CN116409565A - Robot positioning method, device, dispatching equipment, medium and program product - Google Patents

Abstract

Embodiments of the present disclosure provide a robot positioning method, device, scheduling equipment, media, and program products. For the situation where robots form a team and split teams, the method includes: acquiring the first image collected by the first robot at a fixed point in the team, and acquiring The second image collected by the second robot at the fixed point of the team, the first robot is the team leader robot of the original team, and the second robot is the new team leader robot after the team is split; The first target image matched with an image; according to the first image and the second image, it is judged whether the first pose of the second robot at the fixed point of the team is consistent with the second pose of the first robot at the fixed point of the team; if the first pose If it is consistent with the second pose, the positioning of the second robot is performed according to the first target image, which realizes the positioning of the second robot based on the environmental information of the first robot, reduces the data processing amount of the scheduling equipment, and improves the positioning efficiency.

Description

Robot positioning method, device, dispatching equipment, medium and program product

technical field

The present disclosure relates to the technical field of warehousing systems, and in particular to a robot positioning method, device, dispatching equipment, medium and program product.

Background technique

Warehousing is an important link in the logistics process. Robots can replace manual handling of goods and play an important role in intelligent warehousing and logistics.

In order to build a map of the warehouse system or to navigate the robot, it is necessary to position the robot. The commonly used visual positioning technology is SLAM (Simultaneous Localization and Mapping, real-time positioning and map construction) technology. For SLAM, closed-loop detection is an indispensable step for building maps and robot navigation.

When the robot is walking in the warehouse in the form of a fleet, the server or dispatching equipment needs to locate the robot based on the images collected by each robot in the fleet, resulting in many repeated calculation steps in the server or dispatching equipment, resulting in a large computing cost for the server , which leads to high cost and low efficiency of robot positioning.

Contents of the invention

The present disclosure provides a robot positioning method, device, dispatching equipment, media and program products, aiming at the splitting of the robot fleet, by means of image inheritance, omitting the closed-loop operation of the dispatching equipment on the images collected by the second robot at the fixed point of the team The detection step reduces the computing overhead of the scheduling equipment, thereby reducing the cost of robot positioning and improving the efficiency of robot positioning.

In a first aspect, an embodiment of the present disclosure provides a robot positioning method, the method is applied to a scheduling device, and the method includes:

According to the first image collected by the first robot at the fixed point of the team, determine the first target image matching the first image from the environment image library; acquire the second image collected by the second robot at the fixed point of the team; according to the The first image and the second image determine whether the first pose of the second robot at the fixed point of the team is consistent with the second pose of the first robot at the fixed point of the team; if the first If the pose is consistent with the second pose, the positioning of the second robot is performed according to the first target image, so as to update the environment image library or perform navigation of the second robot; wherein, the first A robot and the second robot are located in the first team, and the first robot is the team leader robot of the first team; at the fixed point of the team, the first team is split into the second team and the third team. For the team, the team leader robot of the second team team is the first robot, the team leader robot of the third team team is the second robot, and the robots included in the second team team and the third team team are respectively Quantity is at least 1.

Optionally, the determining the first target image matching the first image from the environment image library according to the first image collected by the first robot at a fixed point of the team includes:

Based on the closed-loop detection algorithm, according to the first image, it is judged whether there is an environmental image matching the first image in the environmental image library; if so, the closed-loop detection is successful, and the matched environmental image is determined as the first image. The first target image of an image.

Optionally, if the first pose is inconsistent with the second pose, the method further includes:

According to the first pose and the second pose, adjust the pose of the second robot; during the pose adjustment of the second robot, obtain at least one frame collected by the second robot Three images; judging whether there is at least one frame of the third image corresponding to the third pose of the second robot corresponding to the first pose; if so, performing the step according to the first target image The positioning of the second robot is used to update the environment image library or perform navigation of the second robot.

Optionally, if the first pose is inconsistent with the second pose, the method further includes:

Determining a second target image matching the second image from the environment image library according to the second image; performing positioning of the second robot according to the second target image.

Optionally, the method also includes:

When determining the fixed point of the team, according to the role of the second robot in the third convoy, determine the sensor that the second robot turns on, wherein the role of the robot in the convoy includes a team leader robot and a team tail robot and robots in the team.

In the second aspect, the embodiment of the present disclosure also provides another robot positioning method, the method is applied to dispatching equipment, and the method includes:

According to the first image collected by the first robot at the fixed point of the team, determine the first target image matching the first image from the environment image library; acquire the state parameters of the first image collected by the first robot, and The first image and the state parameters are sent to a second robot to control the second robot to adjust the state of the second robot based on the state parameters; wherein, the state parameters include The first pose of the first image; when receiving the pose consistent message fed back by the second robot, perform positioning of the second robot according to the first target image to update the environment image library or perform the navigation of the second robot, wherein the pose consistent message is generated when the second robot determines that the pose of the second robot at the fixed point of the team is consistent with the first pose .

Optionally, the state parameters further include a first state parameter, which is used to describe a position parameter of a sensor turned on by the first robot when the first image is captured at a fixed point of the team; The first state parameter is used to control the second robot to turn on the sensor at a corresponding position in the first state parameter, so as to perform image acquisition based on each turned on sensor.

In the third aspect, the embodiment of the present disclosure also provides a robot positioning device, the device is applied to dispatching equipment, and the device includes:

The first image acquisition module is used to acquire the first image collected by the first robot at the fixed point of the team, and the second image collected by the second robot at the fixed point of the team; the first determination module is used to obtain the first image based on the first image , determine the first target image that matches the first image from the environment image library; the pose determination module is used to determine whether the second robot is in the team according to the first image and the second image Whether the first pose of the fixed point is consistent with the second pose of the first robot at the fixed point of the team; the image inherits the first module, and if the first pose is consistent with the second pose, then Perform positioning of the second robot according to the first target image to update the environment image library or perform navigation of the second robot; wherein the first robot and the second robot are located in a first vehicle fleet In, the first robot is the team leader robot of the first team; at the fixed point of the team, the first team is split into the second team and the third team, and the team leader robot of the second team is The first robot, the leader robot of the third team is the second robot, and the number of robots included in the second team and the third team is at least one.

In the fourth aspect, the embodiment of the present disclosure also provides a robot positioning device, the device is applied to dispatching equipment, and the device includes:

The second determining module is used to determine the first target image matching the first image from the environment image library according to the first image collected by the first robot at the fixed point of the team; the state parameter sending module is used to obtain the first target image A robot collects state parameters of the first image, and sends the first image and the state parameters to a second robot, so as to control the second robot to adjust the state of the second robot based on the state parameters ; Wherein, the state parameter includes the first pose when the first robot captures the first image; the image inherits the second module, and is used to receive the pose consistency message fed back by the second robot, Perform positioning of the second robot according to the first target image to update the environment image library or perform navigation of the second robot; wherein, the pose consistency message is for the second robot to determine the Generated by the second robot when the pose of the fixed point of the team is consistent with the first pose.

In a fifth aspect, an embodiment of the present disclosure further provides a scheduling device, including: a memory and at least one processor; the memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory, so that The at least one processor executes the robot positioning method provided in any embodiment of the present disclosure.

In the sixth aspect, the embodiments of the present disclosure also provide a computer-readable storage medium, where computer-executable instructions are stored in the computer-readable storage medium, and when the processor executes the computer-executable instructions, any implementation of the present disclosure can be realized. The robot positioning method provided by the example.

In a seventh aspect, an embodiment of the present disclosure further provides a computer program product, including a computer program, and when the computer program is executed by a processor, the robot positioning method provided in any embodiment of the present disclosure is implemented.

The robot positioning method, device, scheduling equipment, media, and program products provided by the embodiments of the present disclosure are aimed at the situation where a team of robots walking in a team is divided into multiple teams at a fixed point in the team, and the first robot is the team leader of the original robot team. The robot, the second robot is the newly added team leader robot after the split, through the first image and the second image collected by the first robot and the second robot at the fixed point of the team, judge the position of the first robot and the second robot at the fixed point of the team Whether the pose is consistent, if so, based on the result of the closed-loop detection of the first robot, that is, by inheriting the first target image of the first robot, the positioning of the second robot can be performed, thereby omitting the need to perform the positioning of the second robot at the fixed point of the team. The step of closed-loop detection reduces the computing overhead of scheduling equipment and improves the efficiency of robot positioning.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description serve to explain the principles of the disclosure.

FIG. 1 is an application scenario diagram of a robot positioning method provided by an embodiment of the present disclosure;

FIG. 2 is a flowchart of a robot positioning method provided by an embodiment of the present disclosure;

FIG. 3 is a schematic diagram of the splitting team of the robot fleet in the embodiment shown in FIG. 2 of the present disclosure;

FIG. 4 is a flowchart of a robot positioning method provided by another embodiment of the present disclosure;

FIG. 5 is a flowchart of a robot positioning method provided by another embodiment of the present disclosure;

FIG. 6 is a schematic structural diagram of a robot positioning device provided by an embodiment of the present disclosure;

FIG. 7 is a schematic structural diagram of a robot positioning device provided by another embodiment of the present disclosure;

Fig. 8 is a schematic structural diagram of a scheduling device provided by an embodiment of the present disclosure.

By means of the above-mentioned drawings, certain embodiments of the present disclosure have been shown and will be described in more detail hereinafter. These drawings and written description are not intended to limit the scope of the disclosed concept in any way, but to illustrate the disclosed concept for those skilled in the art by referring to specific embodiments.

Detailed ways

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatuses and methods consistent with aspects of the present disclosure as recited in the appended claims.

The technical solution of the present disclosure and how the technical solution of the present disclosure solves the above technical problems will be described in detail below with specific embodiments. The following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present disclosure will be described below with reference to the accompanying drawings.

The application scenarios of the embodiments of the present disclosure are explained below:

Fig. 1 is an application scene diagram of the robot positioning method provided by the embodiment of the present disclosure. As shown in Fig. 1, when constructing a warehouse system map, a robot 110 needs to be positioned. The main process of the positioning technology based on visual SLAM is: The robot 110 uploads the detection images collected by sensors, such as image sensors, inertial sensors, etc., to the dispatching device 120, and the dispatching device 120 determines the camera pose corresponding to each frame detection image based on the detection images received at each time, and based on the image database Each frame detection image is subjected to loopback detection or closed loop detection to determine whether the robot has reached the position when the frame image was collected before, and to locate the robot 110, and then based on the loopback detection result and the camera pose corresponding to each frame detection image, the image The images in the library are optimized to obtain globally consistent trajectories and maps, and realize the construction or optimization of warehouse system maps.

Loop detection is a key step in robot positioning based on SLAM technology. It mainly compares the image currently collected by the robot 110 with the pre-stored environmental image or each frame image in the map to determine whether the robot 110 has arrived at the location before. To locate the robot 110 or adjust the constructed map.

In some embodiments, in order to improve the efficiency of picking and placing goods or building a map, the robot 110 will walk in the form of a convoy, that is, a convoy composed of multiple robots 110 will walk in the form of a robot convoy.

In the robot convoy, the roles of the robot 110 include the team leader robot, the team tail robot and the team robot. The team leader robot is the first robot along the walking direction in the robot team, and the team tail robot is the robot team along the walking direction in the robot team. The last robot, and the robots in the team are the robots located between the tail robot and the team leader robot.

When performing tasks in the form of a robot fleet, such as when performing tasks such as building maps, patrolling, and picking and placing goods, each robot 110 in the robot fleet needs to perform environmental detection based on sensors, and then the dispatching device 120 or server, based on the sensors of the robots The collected data is used for operations such as positioning and obstacle avoidance. When the robots 110 are walking in a convoy, compared with a single robot 110, the amount of data to be processed by the dispatching device 120 or server is doubled.

In order to reduce the data processing tasks of the dispatching device 120, this disclosure aims at splitting the team after the robot team is formed, by inheriting the positioning result or target image of the first robot of the original team, and adding a new one after splitting. The positioning of the team leader robot, that is, the second robot, omits the step of performing loop detection or target image determination on the image collected by the second robot, that is, the second image.

FIG. 2 is a flowchart of a robot positioning method provided by an embodiment of the present disclosure. As shown in FIG. 2 , the robot positioning method is applicable to a storage system and can be executed by a scheduling device. The robot positioning method provided in this embodiment includes the following steps:

Step S201, according to the first image collected by the first robot at the fixed point of the team, determine the first target image matching the first image from the environment image library.

Among them, the fixed point of the team is the place where the team of the first team where the first robot is located is split. The environment image library is each frame image corresponding to the map of the warehouse system that has been established. The first robot is the team leader robot of the first team and the second team, and the team leader robot is the robot that is at the forefront in the walking direction of the team where the robot is located.

In some embodiments, the first convoy is divided into two convoys, the second convoy and the third convoy, the first robot is the leader robot of the second convoy, and the second robot is the leader of the third robot convoy. robot.

In some embodiments, when the number of robots in the convoy where the robot is located is one, the robot is a robot that walks alone.

Specifically, the image with the most features matching the first image or the image with the highest similarity to the first image may be determined from the environment image library by means of feature matching, and be the first target image of the first image.

Specifically, the first target image matching the first image may be determined from the environment image library according to the first image and the odometer corresponding to the first image. Wherein, the odometer corresponding to the first image may be determined according to data collected by sensors such as an inertial sensor and a wheel speedometer of the first robot that collects the first image.

Optionally, according to the first image collected by the first robot at the fixed point of the team, determining the first target image matching the first image from the environment image library includes:

Based on the closed-loop detection algorithm, according to the first image, it is judged whether there is an environmental image matching the first image in the environmental image library; if so, the closed-loop detection is successful, and the matched environmental image is determined as the first image. The first target image of an image.

Among them, the closed-loop detection algorithm can be any algorithm for closed-loop detection, such as BA (Bundle Adjustment, beam adjustment method), bag-of-words model, similarity calculation, deep learning and other algorithms.

Specifically, after the scheduling device obtains the first image, it can perform feature extraction on the first image, and then judge whether there is an environment image matching the first image in the environment image database based on the closed-loop detection algorithm and the characteristics of the first image. If there is an environment image matching the first image, the closed-loop detection is successful, and the environment image matching the first image is output, that is, the first target image.

Specifically, based on the first image and the first target image, that is, the result of the closed-loop detection of the first image, each frame image in the environment image library can be updated to reduce the inter-frame error of the environment image library, thereby reducing the robot's Navigation or positioning errors.

Step S202, acquiring a second image collected by the second robot at a fixed point of the team.

Wherein, the first robot and the second robot are located in the first team, and the first robot is the first robot of the first team; at the fixed point of the team, the first team is split into the first team The second team and the third team, the team leader robot of the second team is the first robot, the team leader robot of the third team is the second robot, the second team and the third team The number of robots included respectively is at least 1.

In some embodiments, the second robot may be any robot in the first fleet except the first robot, such as a team tail robot or a team robot.

In some embodiments, each robot mentioned in the present disclosure, including the first robot and the second robot, is the same robot, the parameters such as the type and size of the robot are the same, and the positions and numbers of the visual sensors on the robot are the same. same.

Specifically, the fixed point of the team is the split position of the first team where the first robot and the second robot are located, and the leader robot of the first team is the first robot. When the first robot and the second robot walk to the fixed point of the team in the form of the first team, the first team is split into two teams, the second team and the third team, and the leader robot of the second team is the first robot , the leader robot of the second team is the second robot, and the walking direction of the second team and the third team at the fixed point of the team are different.

Exemplarily, FIG. 3 is a schematic diagram of the splitting team of the robot fleet in the embodiment shown in FIG. 2 of the present disclosure. As shown in FIG. In the first convoy, the robot 301 is the team leader robot, the robot 302 is the team robot, and the robot 303 is the team tail robot. The robot at the end of the team, that is, the robot 303 needs to leave the original first convoy and walk to the road r1 on the left side of the intersection C1. The robot 301 and the robot 302 will continue to go straight at the intersection C1, that is, walk along the road r2, as shown in Figure 3 The dotted line box is used to represent the robots of the first fleet after the division of the team at the fixed point of the team. The crossroad is a fixed point of the team, the robot 301 is a first robot, and the robot 303 is a second robot.

In some embodiments, at least one vision sensor, such as a laser sensor, an image sensor, etc., may be provided on each side of the robot to collect images of the external environment of the robot. For example, at least one visual sensor can be arranged on each side of the robot chassis, and the visual sensor can also be arranged on the temporary storage shelf or the pick-and-place device of the robot. Vision sensors can be divided into front-view sensors, rear-view sensors, and side-view sensors. Among them, the front-view sensor is used to collect images of the environment in front of the robot in the walking direction, and the rear-view sensor is used to collect images of the environment behind the robot in the walking direction. , and the side-view sensor is used to collect images of the environment on the left or right side of the robot in the walking direction.

In some embodiments, when the robot is walking in the form of a robot convoy or alone, the robot can turn on various vision sensors provided on it, such as the above-mentioned front-view sensor, rear-view sensor and side-view sensor.

Specifically, when the robot is walking in the form of a robot convoy, or when the role of the robot in the robot convoy is updated, the sensors to be turned on by the robot can be determined according to the role of the robot in the robot convoy. The roles of the robots in the fleet include the leader robot, the team robot and the team tail robot, or may include only the team leader robot and the team robot.

In some embodiments, the sensors opened by the team leader robot in the robot fleet can be forward-looking sensors and various side-view sensors, the sensors opened by the robots in the team can be various side-view sensors, and the sensors opened by the team-tail robots can be rear-view sensors. sensors and various side view sensors.

In some embodiments, the sensors activated by the robot at the head of the team can be forward-looking sensors and various side-view sensors, and the sensors activated by the robots in the team and the robots at the end of the team can be rear-view sensors and various side-view sensors.

Specifically, for a robot walking in the form of a convoy or a robot convoy, the sensors turned on by the robot can collect images of the external environment of the robot according to a set cycle. The dispatching device performs steps such as feature extraction, image matching, and odometer calculation for the images of the external environment uploaded by each robot, and performs robot positioning or map reconstruction.

In the first team, since the first robot is located in front of the second robot along the walking direction of the first team, the first robot arrives at the detachment point before the second robot. When the first robot arrives at the fixed point of the team, based on the sensor opened by the first robot at the fixed point of the team, the first image is collected, and the first image is uploaded to the dispatching device; the dispatching device performs positioning of the first robot based on the first image ; And then when the second robot arrives at the fixed point of the team, based on the sensor opened by the second robot at the fixed point of the team, the second image is collected, and the second image is uploaded to the dispatching device.

Specifically, the position corresponding to the image uploaded by the robot can be determined through the inertial sensor, gyroscope, wheel speedometer and other equipment set on the robot (the first robot or the second robot), so as to obtain the image collected by the robot at the fixed point of the team, namely Get the first image or the second image.

Step S203, according to the first image and the second image, determine whether the first pose of the second robot at the fixed point of the team is consistent with the second pose of the first robot at the fixed point of the team .

Among them, the first pose is the pose of the first robot when it collects the first image at the fixed point of the team, and the second pose is the pose of the second robot when it collects the second image at the fixed point of the team.

Specifically, by extracting the image features of the first image and the second image, and based on image feature matching, it is possible to judge whether the first pose of the second robot at the fixed point of the team is consistent with the second pose of the first robot at the fixed point of the team . If the image features of the first image and the second image match or the similarity of the images is greater than a preset value, it is determined that the first pose is consistent with the second pose.

Specifically, based on image features of the same key region in the first image and the second image, it may be determined whether the first pose is consistent with the second pose.

Step S204, if the first pose is consistent with the second pose, perform positioning of the second robot according to the first target image, so as to update the environment image library or perform positioning of the second robot navigation.

Specifically, if the first pose and the second pose are consistent, it can be determined that the orientation of the first robot and the second robot are consistent when they arrive at the fixed point of the team, and then it can be directly based on the positioning result of the first robot or the result of loop closure detection , that is, the first target image performs positioning of the second robot, thereby omitting the process of the scheduling device searching for the second target image matching the second image in the environment image database, and reducing the computing overhead of the scheduling device.

The robot positioning method provided by the embodiments of the present disclosure is aimed at the situation that the team of robots walking in a team is split into multiple teams at the fixed point of the team. The head robot of the team judges whether the poses of the first robot and the second robot at the fixed point of the team are consistent through the first image and the second image collected by the first robot and the second robot at the fixed point of the team. If so, it can be based on the first The result of the closed-loop detection of a robot is to carry out the positioning of the second robot by inheriting the first target image of the first robot, thereby omitting the step of closed-loop detection of the second robot at the fixed point of the team, and reducing the calculation of the dispatching equipment Overhead, improving the efficiency of robot positioning.

In some embodiments, when the second robot reaches the fixed point of the team, it will reduce the walking speed to split the team, such as changing directions. If the first pose is inconsistent with the second pose, the second robot cannot perform positioning of the second robot based on the positioning result of the first robot, that is, the first target image.

Optionally, if the first pose is inconsistent with the second pose, the method further includes:

Determining a second target image matching the second image from the environment image library according to the second image; performing positioning of the second robot according to the second target image.

Specifically, if the first pose is inconsistent with the second pose, it indicates that the pose of the second robot when it reaches the fixed point of the team is different from that of the first robot when it reaches the fixed point of the team. For the collected second image, determine the second target image matching the second image from the environment image library, the specific method is similar to step S201, only the first image is replaced with the second image. Further, according to the second target image, the positioning of the second robot is performed to perform the navigation of the second robot, and based on the second image and the second target image, or based on the result of the closed-loop detection of the second image, each frame in the environment image library is The image is updated to reduce the frame-to-frame error of the environment image library, thereby reducing the robot's navigation or positioning error.

In some embodiments, when the second robot walks to the fixed point of the team, the second robot and the robots behind the second robot along the walking direction may stop walking until receiving a walking control instruction sent by the dispatching device.

Specifically, when the first pose is inconsistent with the second pose, the pose of the second robot at the fixed point of the team can also be adjusted, and one or more frames collected by the second robot when the pose is adjusted at the fixed point of the team can be obtained. Three images, if the third pose of the second robot corresponding to at least one frame of the third image is consistent with the first pose, the positioning of the second robot is performed based on the first target image, and the walking of the second robot is generated based on the positioning result The control instruction is used to control the second robot to walk along the set direction to reach the destination of the second robot or the next path node.

Fig. 4 is a flowchart of a robot positioning method provided by another embodiment of the present disclosure. The robot positioning method provided in this embodiment is based on the embodiment shown in Fig. 2, after step S203, the first digit is added Inconsistency between the second pose and the second pose and adding the step of determining the sensor turned on by the second robot before acquiring the second image, as shown in Figure 4, the robot positioning method provided in this embodiment may include the following steps:

Step S401, acquiring the first image collected by the first robot at the fixed point of the team.

Specifically, the first image can be obtained based on the data collected by the sensors turned on by the first robot when walking to the fixed point of the team, such as the front-view sensor and each side-view sensor.

Step S402, according to the first image, determine a first target image matching the first image from an environment image library.

Step S403, when determining the fixed point of the team, according to the role of the second robot in the third convoy, determine the sensor activated by the second robot.

Among them, the roles of robots in the team include the team leader robot, team tail robot and team robot.

Since the role of the second robot in the team is changed from the tail robot or the team robot in the first team to the team leader robot in the third team at the fixed point of the team, it is necessary to re-determine the second robot’s activation status. For the sensor, specifically, according to the role of the second robot in the third team, that is, the leader robot, the sensors activated by the second robot, such as the front-view sensor and each side-view sensor, can be determined.

Specifically, when the role of each robot in the convoy is changed, the sensors to be turned on by the robot may be determined based on the new role of the robot.

Exemplarily, the sensors opened by the team leader robot in the robot fleet can be forward-looking sensors and various side-view sensors, the sensors opened by the robots in the team can be various side-view sensors, and the sensors opened by the team-tail robots can be rear-view sensors and Individual side view sensors. Alternatively, the sensors activated by the team leader robot in the robot convoy may be front-view sensors and each side-view sensor, and the sensors activated by the team robots and team-tail robots may be various side-view sensors.

Step S404, based on the sensor turned on by the second robot, acquire a second image collected by the second robot at a fixed point of the team.

Specifically, when the second robot walks to the fixed point of the team, the second image of the second robot at the fixed point of the team can be acquired based on the sensors activated by the second robot, such as the front-view sensor and each side-view sensor.

Step S405, according to the first image and the second image, determine whether the first pose of the second robot at the fixed point of the team is consistent with the second pose of the first robot at the fixed point of the team .

Specifically, since the first robot and the second robot have the same role at the fixed point of the team, and both are team leader robots, it can be judged that the first robot and the second robot are in the team by comparing the features of the first image and the second image. Whether the pose at the fixed point, that is, the first pose and the second pose, are consistent.

Specifically, according to the sensors corresponding to the images, the images collected by the sensors of the first robot and the second robot at fixed points in the team can be sequentially judged, that is, the first image and the second image, that is, sequentially comparing I _1i and I _2i , and according to the comparison For the result, judge whether the first pose is consistent with the second pose.

Among them, i=1, 2,..., n, n is the total number of sensors turned on by the team leader robot, I _1i is the first image collected by the i-th turned-on sensor of the first robot, and I _2i is the image of the second robot The second image captured by the i-th turned-on sensor.

Specifically, if I _1i and I _2i , i=1, 2, . . . , n, all match, it is determined that the first pose is consistent with the second pose. Alternatively, if the first image and the second image collected by at least m sensors match, it is determined that the first pose is consistent with the second pose. Wherein, m is a positive integer smaller than n.

Specifically, the first pose of the first robot at the fixed point of the team can be determined according to the first images corresponding to the first robot, and the position of the second robot at the fixed point of the team can be determined based on the second images corresponding to the second robot. second pose. Then judge whether the first pose is consistent with the second pose, such as whether the pose difference between the first pose and the second pose is less than the preset difference, if so, determine that the first pose is consistent with the second pose . Among them, the pose difference can be represented by two parameters, the difference of the heading angle and the difference of the distance from the fixed point of the team.

Step S406, if the first pose is consistent with the second pose, perform positioning of the second robot according to the first target image, so as to update the environment image library or perform positioning of the second robot navigation.

Further, if the pose difference between the first pose and the second pose is too large, such as the difference between the distance from the fixed point of the team is greater than the first difference, or the difference between the orientation angle is greater than the second difference, then according to The second image determines a second target image matching the second image from the environment image library; and performs positioning of the second robot according to the second target image.

Step S407, if the first pose is inconsistent with the second pose, adjust the pose of the second robot according to the first pose and the second pose.

Specifically, when the first pose and the second pose are inconsistent, a pose adjustment command of the second robot can be generated based on the pose difference between the first pose and the second pose, so as to adjust The position and orientation of the second robot to change the pose of the second robot at the fixed point of the team.

Specifically, a pose adjustment instruction of the second robot may also be generated according to the difference in image coordinates of corresponding points in the first image and the second image, so as to adjust the pose of the second robot based on the pose adjustment instruction.

Further, when the second pose is inconsistent with the first pose, the type and task priority of the tasks performed by the second robot or each robot in the third fleet are obtained, and when each of the second robot or the third fleet When the type of task performed by the robot and the priority of the task meet the preset conditions, a pose adjustment instruction of the second robot is generated according to the first pose and the second pose. If the type of tasks performed by the second robot or each robot in the third fleet and the task priority do not meet the preset conditions, then determine a second target image matching the second image from the environment image library according to the second image, Positioning of the second robot is performed according to the second target image.

Exemplarily, the preset condition may be that the type of the task is a set type, and the priority of the task is lower than the preset priority.

Step S408, acquiring at least one frame of a third image captured by the second robot while the second robot is performing pose adjustment.

Specifically, the specific manner of acquiring the third image is similar to that of the second image, and only the acquisition time is different, which will not be repeated here.

Step S409, judging whether the third pose of the second robot corresponding to at least one frame of the third image is consistent with the first pose.

Specifically, feature extraction can be performed on the third image, and based on the features corresponding to the third image, the third pose of the second robot corresponding to each frame of the third image is determined, and then it is judged whether there is a third pose that is different from the first robot in each third pose. A consistent pose.

Step S410, if it exists, perform positioning of the second robot according to the first target image, so as to update the environment image library or perform navigation of the second robot.

Specifically, if there is at least one third pose corresponding to the third image that is consistent with the first pose, the second robot can be positioned based on the first target image of the first robot, that is, the second robot inherits the position of the first robot. The results of loopback detection or positioning eliminate the need to search for a target image that matches the second image from an environment image library that includes many frames of images, saving computing resources for dispatching equipment and improving the positioning efficiency of the second robot at the fixed point of the team.

Further, if there is no third pose corresponding to the third image that is consistent with the first pose, then according to the second image or the third image, determine the second pose that matches the second image or the third image from the environment image library. target image; performing positioning of the second robot according to the second target image.

In this embodiment, for the situation that the team of robots walking in teams is divided into multiple teams at fixed points, based on the role of the second robot in the new team, that is, the third team obtained after the split of the first team, Automatically control the sensor opened by the second robot, and then based on the second image collected by the opened sensor at the fixed point of the team and the first image collected by the first robot of the original team at the fixed point of the team, it is judged that the second robot is in the same position as the first robot. Whether the poses and poses of the team’s fixed-point acquisition images are consistent, if they are consistent, the second robot will inherit the matching results or loop-back detection results of the first robot, and perform positioning of the second robot based on the first target image of the first robot; if they are not consistent, Then adjust the pose of the second robot at the fixed point of the team to obtain a new image collected by the second robot at the fixed point of the team, that is, the third image, so as to judge whether the pose of the second robot corresponding to the third image is consistent with the first The poses and poses of the first robot corresponding to the image are consistent. If they are consistent, the second robot will inherit the matching result or loop detection result of the first robot, and locate the second robot based on the first target image of the first robot. The inheritance of image matching results of robots with the same pose and role at a fixed point reduces the amount of data processing of scheduling equipment and improves the efficiency of robot positioning.

FIG. 5 is a flowchart of a robot positioning method provided in another embodiment of the present disclosure. The robot positioning method provided in this embodiment is executed by a scheduling device. As shown in FIG. 5 , the robot positioning method provided in this embodiment includes the following steps:

Step S501, according to the first image collected by the first robot at the fixed point of the team, determine the first target image matching the first image from the environment image library.

Step S502, obtaining the state parameters of the first image collected by the first robot, and sending the first image and state parameters to the second robot, so as to control the second robot to adjust the second robot based on the state parameters. The state of the robot.

Wherein, the state parameter includes a first pose of the first robot when capturing the first image.

Specifically, after the team collects the first image at a fixed point, the first robot may send the state parameters when the first image is collected and the first image to the dispatching device. The dispatching device sends the state parameters and the first image to the second robot, so that the second robot adjusts the posture of the second robot at the fixed point of the team based on the state parameters, collects the second image, and judges the collected second image Whether it matches with the first image, if the similarity is greater than the preset value, if so, determine that the pose of the second robot at the fixed point of the team is consistent with the first pose of the first robot, and generate a pose consistent message.

Specifically, the second robot can judge whether the second image collected by the second robot at the fixed point of the team matches the first image; if they match, then determine that the current pose of the second robot at the fixed point of the team is consistent with the first pose; if not match, then fine-tune the pose of the second robot, or adjust the pose of the second robot based on the pose difference between the current pose and the first pose, and collect new images during the pose adjustment of the second robot, Determine whether there is an image matching the first image in the new image, and if so, determine that the pose of the second robot is consistent with the first pose.

Optionally, the state parameters further include a first state parameter, which is used to describe a position parameter of a sensor turned on by the first robot when the first image is captured at a fixed point of the team; The first state parameter is used to control the second robot to turn on the sensor at a corresponding position in the first state parameter, so as to perform image acquisition based on each turned on sensor.

Specifically, when the second robot walks to the fixed point of the team, the sensor of the second robot can be turned on based on the first state parameter among the state parameters, so that the second robot is consistent with the sensor turned on by the first robot at the fixed point of the team. It is the front-view sensor and each side-view sensor; and then the acquisition of the second image is performed based on each sensor that is turned on.

Step S503, when receiving the pose consistent message fed back by the second robot, perform positioning of the second robot according to the first target image, so as to update the environment image library or perform positioning of the second robot navigation.

Wherein, the pose consistency message is generated when the second robot determines that the pose of the second robot at the fixed point of the team is consistent with the first pose.

Specifically, when the second robot determines that the image collected by the second robot at the fixed point of the team matches the first image or the similarity is greater than a preset value by means of image feature matching, it is determined that the pose of the second robot at the fixed point of the team is consistent with the first image. If a pose is consistent, generate a pose consistent message, and send the pose consistent message to the scheduling device. After the dispatching device receives the pose consistent message fed back by the second robot, it directly performs the positioning of the second robot based on the first target image of the positioning result of the first robot, thereby omitting the need to search the environment image library for matching with the second image. The process of the image, that is, the process of omitting the closed-loop detection of the second image, reduces the calculation amount of the scheduling equipment, and improves the positioning efficiency of the second robot at the fixed point of the team.

In this embodiment, for the situation where the robot convoy walking in teams is split into multiple convoys at fixed points, the first robot is the leader robot of the original robot convoy, and the second robot is the newly added leader robot after the split. , based on the state parameters of the first robot when collecting the first image at the fixed point of the team, the second robot is controlled to collect the second image with the same state parameters at the fixed point of the team, and then the second robot judges the first image and the second image Whether the image features match, that is, to judge whether the poses of the first robot and the second robot at the fixed point of the team are consistent. Robot positioning, thereby omitting the steps of closed-loop detection or image matching for the second robot at the fixed point of the team, reducing the computing overhead of the dispatching equipment, and improving the efficiency of robot positioning.

FIG. 6 is a schematic structural diagram of a robot positioning device provided by an embodiment of the present disclosure. As shown in FIG. The module 630 and the image inherit from the first module 640 .

Wherein, the first determination module 610 is used to determine the first target image matching the first image from the environment image library according to the first image collected by the first robot at the fixed point of the team; the image acquisition module 620 uses Obtaining the second image collected by the second robot at the fixed point of the team; the pose determination module 630 is used to determine the second image of the second robot at the fixed point of the team according to the first image and the second image. Whether a pose is consistent with the second pose of the first robot at the fixed point of the team; the image inherits the first module 640, for if the first pose is consistent with the second pose, then according to the The first target image is used to perform positioning of the second robot, so as to update the environment image library or perform navigation of the second robot; wherein, the first robot and the second robot are located in the first fleet, The first robot is the team leader robot of the first team; the first team is split into the second team and the third team at the fixed point of the team, and the team leader robot of the second team is the team robot. The first robot, the leader robot of the third team is the second robot, and the number of robots included in the second team and the third team is at least one.

Optionally, the first determining module 610 is specifically configured to:

Based on the closed-loop detection algorithm, according to the first image, it is judged whether there is an environmental image matching the first image in the environmental image library; if there is an environmental image matching the first image in the environmental image library, Then the closed loop detection is successful, and the matched environment image is determined as the first target image of the first image.

Optionally, the device also includes:

a pose adjustment module, configured to adjust the pose of the second robot according to the first pose and the second pose if the first pose is inconsistent with the second pose; the third The image acquisition module is used to acquire at least one frame of the third image collected by the second robot during the pose adjustment of the second robot; the pose consistency judgment module is used to judge whether there is at least one frame of the The third pose of the second robot corresponding to the third image is consistent with the first pose; the image inherits the third module, and is used for if there is at least one frame of the second robot corresponding to the third image If the third pose of the object is consistent with the first pose, the positioning of the second robot is performed according to the first target image, so as to update the environment image library or perform navigation of the second robot.

Optionally, the device also includes:

A second target image determining module, configured to determine a second object matching the second image from the environment image library according to the second image if the first pose is inconsistent with the second pose. Target image; a second robot positioning module, configured to perform positioning of the second robot according to the second target image.

Optionally, the device also includes:

A sensor activation module, configured to determine the sensor activated by the second robot according to the role of the second robot in the third convoy when determining the fixed point of the team, wherein the role of the robot in the convoy includes team Robots at the head of the team, robots at the end of the team, and robots in the team.

The robot positioning device provided in the embodiment of the present disclosure can execute the robot positioning method provided in the embodiment corresponding to FIG. 2 or FIG. 4 of the present disclosure, and has corresponding functional modules and beneficial effects for executing the method.

FIG. 7 is a schematic structural diagram of a robot positioning device provided by another embodiment of the present disclosure. As shown in FIG. Inherit the second module 730 .

Among them, the second determination module 710 is used to determine the first target image matching the first image from the environment image library according to the first image collected by the first robot at the fixed point of the team; the state parameter sending module 720 is used to Obtaining the state parameters of the first image captured by the first robot, and sending the first image and the state parameters to a second robot, so as to control the second robot to adjust the first image based on the state parameters. The state of the second robot; wherein, the state parameters include the first pose of the first robot when the first image is captured; the image inherits the second module 730, which is used to receive the position fed back by the second robot When the pose consistent message is used, the positioning of the second robot is performed according to the first target image, so as to update the environment image library or perform the navigation of the second robot; wherein, the pose consistent message is the first target image The second robot determines that the pose of the second robot at the fixed point of the team is consistent with the first pose.

Optionally, the second determination module 710 is specifically used for:

Based on the closed-loop detection algorithm, according to the first image, it is judged whether there is an environmental image matching the first image in the environmental image library; if there is an environmental image matching the first image in the environmental image library, Then the closed loop detection is successful, and the matched environment image is determined as the first target image of the first image.

Optionally, the state parameters further include a first state parameter, which is used to describe a position parameter of a sensor turned on by the first robot when the first image is captured at a fixed point of the team; The first state parameter is used to control the second robot to turn on the sensor at a corresponding position in the first state parameter, so as to perform image acquisition based on each turned on sensor.

The robot positioning device provided in the embodiment of the present disclosure can execute the robot positioning method provided in the embodiment corresponding to FIG. 5 of the present disclosure, and has corresponding functional modules and beneficial effects for executing the method.

FIG. 8 is a schematic structural diagram of a scheduling device provided by an embodiment of the present disclosure. As shown in FIG. 8 , the scheduling device includes: a memory 810, a processor 820, and a computer program.

Wherein, the computer program is stored in the memory 810 and is configured to be executed by the processor 820 to implement the robot positioning method provided in any one of the embodiments corresponding to FIG. 2 , FIG. 4 and FIG. 5 of the present disclosure.

Wherein, the memory 810 and the processor 820 are connected through a bus 830 .

Relevant descriptions can be understood by referring to the relevant descriptions and effects corresponding to the steps in FIG. 2 , FIG. 4 , and FIG. 5 , and details are not repeated here.

An embodiment of the present disclosure provides a storage system, and the storage system includes: the scheduling device and the robot provided in the embodiment corresponding to FIG. 8 .

In some embodiments, the storage system further includes storage shelves for storing goods or material boxes.

In some embodiments, the storage system further includes an operation platform and transfer devices such as an unloader, an elevator, and a transportation line. The robot is used to transport the goods or material boxes to the operation platform or the transfer device connected with the operation platform to complete the order corresponding to the operation platform.

An embodiment of the present disclosure provides a computer-readable storage medium on which a computer program is stored, and the computer program is executed by a processor to implement any of the embodiments corresponding to FIG. 2 , FIG. 4 and FIG. 5 of the present disclosure. robot localization method.

Among them, the computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device and the like.

The present disclosure also provides a program product including an executable computer program stored in a readable storage medium. At least one processor of the scheduling device or the storage system can read the computer program from the readable storage medium, and the at least one processor executes the computer program to make the robot positioning device implement the robot positioning method provided in the above-mentioned various embodiments.

In the several embodiments provided in the present disclosure, it should be understood that the disclosed devices and methods may be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the modules is only a logical function division. In actual implementation, there may be other division methods, for example, multiple modules can be combined or integrated. to another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or modules may be in electrical, mechanical or other forms.

The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional module in each embodiment of the present disclosure may be integrated into one processing unit, each module may exist separately physically, or two or more modules may be integrated into one unit. The units formed by the above modules can be implemented in the form of hardware, or in the form of hardware plus software functional units.

The above-mentioned integrated modules implemented in the form of software function modules can be stored in a computer-readable storage medium. The above-mentioned software function modules are stored in a storage medium, and include several instructions to enable a computer device (which may be a personal computer, server, or network device, etc.) or a processor (English: processor) to execute the functions described in various embodiments of the present disclosure. part of the method.

It should be understood that the above-mentioned processor may be a central processing unit (Central Processing Unit, referred to as CPU), and may also be other general-purpose processors, a digital signal processor (Digital Signal Processor, referred to as DSP), an application specific integrated circuit (Application Specific Integrated Circuit, referred to as ASIC) and so on. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like. The steps of the method disclosed in conjunction with the present disclosure may be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor.

The storage may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk storage, and may also be a U disk, a mobile hard disk, a read-only memory, a magnetic disk, or an optical disk.

The bus may be an Industry Standard Architecture (Industry Standard Architecture, ISA for short) bus, a Peripheral Component Interconnect (PCI for short) bus, or an Extended Industry Standard Architecture (EISA for short) bus. The bus can be divided into address bus, data bus, control bus and so on. For ease of representation, the buses in the drawings of the present disclosure are not limited to only one bus or one type of bus.

The above-mentioned storage medium can be realized by any type of volatile or non-volatile storage device or their combination, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable In addition to programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.

An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. Of course, the storage medium may also be a component of the processor. The processor and the storage medium may be located in application specific integrated circuits (Application Specific Integrated Circuits, ASIC for short). Of course, the processor and the storage medium can also exist in the electronic device or the main control device as discrete components.

Those of ordinary skill in the art can understand that all or part of the steps for implementing the above method embodiments can be completed by program instructions and related hardware. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps including the above-mentioned method embodiments; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other various media that can store program codes.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present disclosure, not to limit them; although the present disclosure has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: It is still possible to modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the various embodiments of the present disclosure. scope.

Claims (12)

1. A robot positioning method, the method being applied to a scheduling device, the method comprising:

determining a first target image matched with a first image from an environment image library according to the first image acquired by a first robot at a team fixed point;

Acquiring a second image acquired by a second robot at the team fixed point;

judging whether a first pose of the second robot at the team fixed point is consistent with a second pose of the first robot at the team fixed point or not according to the first image and the second image;

if the first pose is consistent with the second pose, positioning the second robot according to the first target image so as to update the environment image library or navigate the second robot;

the first robot and the second robot are located in a first motorcade, and the first robot is a first motorcade robot of the first motorcade; the first motorcade is split into a second motorcade and a third motorcade at the grouping fixed point, the first motorcade robot of the second motorcade is the first robot, the first motorcade robot of the third motorcade is the second robot, and the number of robots respectively included by the second motorcade and the third motorcade is at least 1.

2. The method of claim 1, wherein determining a first target image from an environmental image library that matches the first image from the first image acquired by the first robot at the team fixed point comprises:

Based on a closed loop detection algorithm, judging whether an environment image matched with the first image exists in the environment image library according to the first image;

if yes, the closed loop detection is successful, and the matched environment image is determined to be a first target image of the first image.

3. The method of claim 1, wherein if the first pose is not consistent with the second pose, the method further comprises:

according to the first pose and the second pose, adjusting the pose of the second robot;

acquiring at least one frame of third image acquired by the second robot during the pose adjustment of the second robot;

judging whether a third pose of the second robot corresponding to at least one frame of the third image is consistent with the first pose;

and if the environment image exists, positioning the second robot according to the first target image so as to update the environment image library or navigate the second robot.

4. The method of claim 1, wherein if the first pose is not consistent with the second pose, the method further comprises:

determining a second target image matched with the second image from the environment image library according to the second image;

And positioning the second robot according to the second target image.

5. The method according to any one of claims 1-4, further comprising:

and when the grouping fixed point is determined, determining a sensor started by the second robot according to the role of the second robot in the third vehicle team, wherein the roles of the robots in the vehicle team comprise a first robot, a tail robot and a robot in the vehicle team.

6. A robot positioning method, the method being applied to a scheduling device, the method comprising:

determining a first target image matched with a first image from an environment image library according to the first image acquired by a first robot at a team fixed point;

acquiring state parameters of the first image acquired by the first robot, and sending the first image and the state parameters to a second robot so as to control the second robot to adjust the state of the second robot based on the state parameters; wherein the state parameter comprises a first pose of the first robot when the first robot acquires the first image;

and when a pose consistency message fed back by the second robot is received, positioning the second robot according to the first target image so as to update the environment image library or navigate the second robot, wherein the pose consistency message is generated when the pose of the second robot at the team fixed point is determined to be consistent with the first pose for the second robot.

7. The method of claim 6, wherein the status parameters further comprise a first status parameter describing a position parameter of a sensor that the first robot turned on when the first image was acquired at the enqueue setpoint; the first state parameters are used for controlling the second robot to start the sensors at the corresponding positions in the first state parameters so as to acquire images based on the started sensors.

8. A robotic positioning device, the device being applied to a scheduling apparatus, the device comprising:

the first determining module is used for determining a first target image matched with the first image from an environment image library according to the first image acquired by the first robot at the team fixed point;

the image acquisition module is used for acquiring a second image acquired by a second robot at the team fixed point;

the pose judgment module is used for judging whether the first pose of the second robot at the team fixed point is consistent with the second pose of the first robot at the team fixed point according to the first image and the second image;

the image inheritance first module is used for positioning the second robot according to the first target image if the first pose is consistent with the second pose so as to update the environment image library or navigate the second robot;

The first robot and the second robot are located in a first motorcade, and the first robot is a first motorcade robot of the first motorcade; the first motorcade is split into a second motorcade and a third motorcade at the grouping fixed point, the first motorcade robot of the second motorcade is the first robot, the first motorcade robot of the third motorcade is the second robot, and the number of robots respectively included by the second motorcade and the third motorcade is at least 1.

9. A robotic positioning device, the device being applied to a scheduling apparatus, the device comprising:

the second determining module is used for determining a first target image matched with the first image from an environment image library according to the first image acquired by the first robot at the team fixed point;

the state parameter sending module is used for acquiring state parameters of the first image acquired by the first robot and sending the first image and the state parameters to a second robot so as to control the second robot to adjust the state of the second robot based on the state parameters; wherein the state parameter comprises a first pose of the first robot when the first robot acquires the first image;

The image inheritance second module is used for positioning the second robot according to the first target image when receiving the pose consistency message fed back by the second robot so as to update the environment image library or navigate the second robot;

and the pose consistency message is generated when the second robot determines that the pose of the team fixed point is consistent with the first pose for the second robot.

10. A scheduling apparatus, comprising:

a memory and at least one processor;

the memory stores computer-executable instructions;

the at least one processor executing computer-executable instructions stored in the memory, causing the at least one processor to perform the robotic positioning method of any one of claims 1-7.

11. A computer readable storage medium, wherein computer executable instructions are stored in the computer readable storage medium, which when executed by a processor, implement the robot positioning method according to any of claims 1-7.

12. A computer program product comprising a computer program, characterized in that the computer program, when being executed by a processor, implements the robot positioning method according to any of claims 1-7.

Images