JP2024082211A - Robot control system, robot control program - Google Patents

Abstract

To perform a series of work of checking a position of a package, gripping, and holding by a holding part quickly and reliably.SOLUTION: A robot control system classifies a package 100 on the basis of its properties (a shape and weight etc.) and selects a gripping part 20 and a suction pad 24 on the basis of the classified result to allow a single humanoid robot 1 to pick up various types of packages 100. In other words, it is not necessary to select a different robot to match for the properties of the package 100 and thus efficiency of picking work can be improved.SELECTED DRAWING: Figure 10

Description

The present invention relates to a robot control system and a robot control program.

Humanoid robots are used in factory production lines to perform tasks automatically. Patent Document 1 describes posture control of humanoid robots.

Patent Document 2 also describes a robot arm having multiple joints that is driven by an elastic actuator, and is controlled by a hand support member that is disposed at the hand of the robot arm and supports the robot arm by contacting a support surface, and a control unit that controls the contact force between the hand support member and the support surface and at the same time controls the position and posture of the hand of the robot arm.

JP 2019-093506 A WO2011/001569 Publication

However, conventional picking tasks in warehouses using humanoid robots currently rely on human hands when picking up items (shampoo, conditioner, cosmetics, toothpaste, instant ramen, snack wrappers, and other items of different shapes, weights, hardness, and fragility) from shelves on which they are displayed and then packing them into the appropriate packaging (boxes, etc.).

Even if we try to make the structure of the robot's gripping part a finger type, the fingers and arms move slowly, which results in low productivity.

In consideration of the above, the present invention aims to provide a robot control system and a robot control program that can quickly and reliably perform a series of tasks, including checking the position of a package, grabbing it, and holding it, using the gripping unit.

The control system for a robot according to the present invention is a control system for a robot capable of grasping an object using a first gripping unit having a palm portion that serves as a base for grasping an object and a plurality of finger portions attached to the palm portion, and a second gripping unit having a holder capable of grasping and holding the object, the control system comprising: a plurality of types of suction pads selectively provided for each of the palm portion and finger portions of the first gripping unit and capable of suctioning and grasping the object with different suction areas; a sensor unit provided on the first gripping unit and the second gripping unit that detects at least shape information of the object; and a weight information of the object based on previously registered weight information of the object. The device has a selection unit that selects, as a function for gripping the object, a first function for gripping with the first gripping unit, a second function for gripping with the second gripping unit, or a third function for using the first and second functions in combination; a control unit that controls the operation of gripping the object based on the function selected by the selection unit; a determination unit that determines whether or not shape information of the object detected by the sensor unit during an approach operation to the object in a gripping operation by the control unit is within an allowable range for the function for gripping the object selected by the selection unit; and a change unit that changes the function for gripping the object when the determination unit determines that the shape information is outside the allowable range.

According to the present invention, the selection unit selects, based on preregistered weight information of the object, a function for gripping the object from among a first function for gripping with a first gripping unit, a second function for gripping with a second gripping unit, and a third function for using both the first and second functions.

The control unit controls the action of grasping the object based on the function selected by the selection unit.

Here, the determination unit determines whether the shape information of the object detected by the sensor unit during the control unit's approach to the object in the gripping operation is within the allowable range of the function for gripping the object selected by the selection unit, and the change unit changes the function for gripping the object when the determination unit determines that it is outside the allowable range.

This allows the gripping unit to quickly and reliably perform the series of tasks of checking the location of the luggage, grabbing it, and holding it.

The present invention is characterized in that the determination unit makes a determination based on a table in which the function of grasping the object is associated with the acceptable range of shape information of the object.

By using a table that associates the ability to grasp an object with the acceptable range of the object's shape information in advance, it is easy to determine whether the object is within the acceptable range of grasping based on the shape information.

In the present invention, the sensor unit is characterized by having a camera that takes an image of the object and identifies the type of the object, and a motion processing unit that identifies the position of the object.

Based on the detection results from the sensor unit (object information including the type, shape, or size of the object, and its position), it is possible to determine whether the object complies with the standard.

The camera identifies the photographed object (hereinafter sometimes referred to as luggage) based on the captured image information. In other words, its role is to obtain information to identify the type of object (shape, size, hardness, etc.).

The motion processing unit (MoPU) outputs vector information of the movement of a point indicating the location of an object along a specified coordinate axis as position information together with the movement information. In other words, the movement information output from the MoPU contains only information indicating the movement (movement direction and movement speed) on the coordinate axes (x-axis, y-axis, z-axis) of the center point (or center of gravity) of the object. In other words, it is possible to provide accurate guidance on the trajectory of the gripping unit as it approaches the object.

In the present invention, the first gripping section has three fingers, a suction pad having a smaller diameter than the other suction pads is attached to the tip of the first finger, and a suction pad having a larger diameter than the other suction pads is attached to the tip of the second finger and the third finger, and the control section classifies the object into extra-large, large, medium, small, and extra-small sizes as weight classification sizes based on the weight information, and when the object is the extra-large size, the control section grasps the object using a holder provided on the second gripping section, and holds the object. However, when the object is large, the suction pad provided on the palm is used to grasp the object; when the object is medium, the large-diameter suction pad provided on the tip of the second finger and the third finger is used to grasp the object; when the object is small, the large-diameter suction pad provided on the tip of the second finger or the third finger is used to grasp the object; and when the object is extra small, the small-diameter suction pad provided on the tip of the first finger is used to grasp the object.

Since each robot has a gripping function that matches the attributes of the target part, there is no need to select a robot, which makes picking work more efficient.

The robot control system according to the present invention is characterized in that it operates a computer as the above-mentioned selection unit, control unit, determination unit, and change unit.

Note that the above summary of the invention does not list all of the necessary features of the present invention. Also, subcombinations of these features may also be inventions.

As described above, the present invention has the advantage that the gripping unit can quickly and reliably perform a series of tasks including checking the position of the luggage, grabbing it, and holding it.

1A is a perspective view of a storage base according to the present embodiment, and FIG. 1B is an enlarged perspective view of a storage space portion of a shelf. FIG. 1 is a front view of a humanoid robot according to an embodiment of the present invention. FIG. 1 is a side view of a humanoid robot according to an embodiment of the present invention. 1A is a front view of the palm side of a grip part according to the present embodiment, and FIG. 1B is a perspective view of a suction pad attached to the grip part. 1A is a perspective view of a left-hand grip portion according to the present embodiment, and FIG. 1B is a perspective view of a right-hand grip portion according to the present embodiment. FIG. 2 is a diagram illustrating an example of a functional configuration of a humanoid robot. FIG. 2 is a diagram illustrating an example of a processing routine executed by the information processing device. 6 is a flowchart showing a procedure of gripping control when gripping an object by a gripper in conjunction with the overall movement of the humanoid robot in FIG. 5 . 9 is a control flowchart showing a selection process routine for a gripper and a suction pad in step 160 of FIG. 8 . FIG. 11 is a functional block diagram for controlling work related to shipping and receiving operations executed in cooperation with a storage base management server and each information processing device of the humanoid robot. 13 is a flowchart showing a main routine of a storage-based management server for controlling work instructions based on a work schedule. FIG. 1 is a diagram illustrating an example of computer hardware that functions as an information processing device.

The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

(Logistics Center Configuration)
FIG. 1A is a perspective view of a storage base 50 which is a warehouse in a logistics center according to this embodiment and which stores packages 100 (see FIG. 1B).

A number of shelves 52 (some with indicators) are arranged on the storage base 50. As an example, six shelves 52 are arranged in a row, and two rows form a so-called "island." The formation of islands and the number of shelves 52 are not limited to the arrangement in FIG. 1 (A).

As shown in FIG. 1B, the shelf 52 is provided with a plurality of storage spaces 52A, and each storage space 52A is capable of storing luggage 100.

Multiple conveying devices 54 are arranged within the storage base 50 so as to be movable within the storage base 50.

The operation of the transport device 54 is controlled by commands from a storage base management server 56, which collectively manages the controlled devices in the storage base 50, and the transport device 54 moves, for example, between a predetermined waiting position, an arrangement position on the shelf 52, and a picking station 58.

When the transport device 54 faces the shelf 52, it can enter the gap under the shelf 52 and lift it up, lifting it off the floor of the storage base 50. Therefore, the shelf 52 can be moved to a desired position by moving the transport device 54.

A plurality of picking stations 58 are arranged at predetermined positions on the storage base 50. In FIG. 1(A), two picking stations 58 are illustrated, but one or three or more picking stations 58 may be used. A humanoid robot 1 is arranged at each picking station 58. The humanoid robot 1 may be used instead of the transport device 54. The humanoid robot 1 can perform operations such as transporting a shelf 52, directly removing luggage 100 from the shelf 52 (storage space 52A), or directly storing luggage in the shelf 52 (storage space 52A). Details of the humanoid robot 1 will be described later.

The humanoid robot 1 performs the tasks it receives commands to perform under the management of the storage base management server 56, in cooperation with the information processing device 14 (see Figure 6) installed in the humanoid robot 1.

In other words, at each picking station 58, when the transport device 54 places the shelf 52 at the entrance 58A of the picking station 58, the humanoid robot 1 receives commands from the storage base management server 56 and performs the storing and unstoring operations of the luggage 100.

The warehousing and shipping operations themselves are controlled by an information processing device 14 (see Figure 6) that each humanoid robot 1 possesses.

The picking station 58 has an opening 58A where the shelves 52 are arranged, as well as a work space 58B where sorting and packaging are performed.

The shipping process involves removing packages 100 stored on shelves 52 according to their destination, sorting them by destination, and packing them into boxes 60 (transportation materials) for each destination.

The warehousing operation involves sorting the luggage 100 that arrives at the warehouse into the storage shelves 52 and placing the sorted luggage 100 in the designated positions on the shelves 52.

In this embodiment, the tasks in the shipping process and the tasks in the receiving process are defined as follows:

(Warehouse Delivery Operation 1) Picking Operation This is an operation in which the humanoid robot 1 picks up a specified piece of luggage 100 from the shelf 52 that has arrived at the entrance 58A and moves it to the work space 19.

(Warehouse Delivery Operation 2) Assortment Work This is the work of storing the luggage 100 taken out to the work space 58B into boxes 60 according to the destination, and packaging the boxes, etc.

(Warehouse Delivery Operation 3) Warehouse Delivery Operation This is the operation of requesting the next shelf 52 after the picking operation for the shelf 52 arranged at the frontage 58A has been completed.

(Warehouse Operation 1) Picking Work This is the work of removing the specified cargo 100 from the truck or pallet that has arrived at the entrance 58A and moving it to the work space 58B.

(Warehouse Operation 2) Assortment Work This is the work of storing the luggage 100 taken out to the work space 58B on the shelf 52 according to the storage destination.

(Warehouse Receiving Operation 3) Warehouse Receiving Operation This is the operation to request the next shelf 52 after the assortment operation on the shelf 52 arranged at the frontage 58A is completed.

Here, information regarding the work of each humanoid robot 1 is sequentially sent from the information processing device 14 of the humanoid robot 1 to the storage base management server 56. As a result, the storage base management server 56 collectively manages the operating status (work progress information) of multiple humanoid robots 1.

(Detailed Configuration of Humanoid Robot 1)
Fig. 2 is a front view of the humanoid robot 1 according to the present embodiment. As shown in Fig. 2, the humanoid robot 1 according to the present embodiment includes an upper body 2, legs 3, and a connecting part 4 that rotatably connects the upper body 2 to the legs 3, and is arranged, for example, on a storage base shown in Fig. 1(A) or a production line in a factory, and performs work on a line including a shelf 52 on which packages 100 (see Fig. 2(B)) that are picking objects are displayed, or on an object (such as a fallen object) on the floor. Note that the work includes not only picking, which is to grip the package 100, but also packing, which is to store the gripped package 100 in a predetermined case (such as the box 60 shown in Fig. 1(A)).

The upper body 2 has two arms 5 and 6. The arms 5 and 6 are attached to the left and right of the upper body 2 so that they can rotate freely. In addition, a gripping part 20 (described in detail later) for gripping luggage 100 is attached to the tip of the arms 5 and 6. The number of arms is not limited to two, and may be one or three or more.

The leg 3 has two wheels 7 and 8 attached to its bottom, allowing it to move across the floor on which the humanoid robot 1 is placed.

The connecting part 4 connects the upper body part 2 and the legs 3 in a rotatable manner. Therefore, the upper body part 2 can lean forward and backward with respect to the legs 3. Therefore, the humanoid robot 1 according to this embodiment can lean the upper body part 2 forward with respect to the legs 3 as shown in FIG. 3, and pick up luggage 100 placed on a shelf 52, luggage 100 placed on the floor, and luggage 100 that has fallen to the floor during work.

The legs 3 have a balancing function to prevent the humanoid robot 1 from falling over when the upper body 2 leans forward or backward relative to the legs 3 or when the humanoid robot 1 moves.

The connecting part 4 also has a function that allows the distance between the upper body part 2 and the legs 3 to be changed, as shown in FIG. 2. Therefore, the vertical position of the upper body part 2 relative to the legs 3 can be adjusted, as shown by arrow A, to match the height of the workbench on the production line.

In addition, the humanoid robot 1 according to this embodiment is controlled by a control system 10 implemented within the humanoid robot 1.

(Structure of grip portion 20)
As shown in Fig. 4A, the gripping parts 20 attached to the ends of the arms 5 and 6 have a structure similar to that of a human hand (Intelligent Hand System) on one side (the arm 6 side, which is the left arm in this embodiment) and a rectangular structure on the other side (the arm 5 side, which is the right arm in this embodiment). Hereinafter, when distinguishing between the left and right gripping parts 20, the left hand side will be referred to as "grip 20L" and the right hand side will be referred to as "grip 20R"). The gripping parts 20 are attached to the arms 5 and 6 so as to be freely rotatable.

(Left hand grip 20L)
4A and 5A, the grip portion 20L according to the present embodiment has three fingers 22A, 22B, and 22C, each of which has a plurality of joints. Note that, although the number of fingers of the grip portion 20L is three in the present embodiment, it may have a multi-finger structure, such as five fingers.

Multiple suction pads 24 (four in this embodiment) are attached to the palm side 20A of the gripping portion 20L.

In addition, in this embodiment, a palm sensor 26 is attached to the center of the palm side 20A of the gripping portion 20L. The palm sensor 26 is equipped with a high-resolution camera that identifies the type of luggage 100, and a MoPU (Motion Processing Unit) that identifies the position of the luggage 100.

In addition, suction pads 24X and 24Y are attached to the tips of the three fingers 22A, 22B, and 22C, respectively. The classification of suction pads 24X and 24Y will be described later.

(Right hand grip 20R)
4(A) and 5(B), the grip part 20R according to this embodiment has a rectangular main body part 20B, and a plurality of (four in this embodiment) suction pads 24 are attached to the tip surface of the main body part 20B. The suction pads 24 may be either the suction pad 24X or the suction pad 24Y used in the left grip part 20L, or may have a different diameter dimension.

In addition, in this embodiment, a palm sensor 26 is attached to the center of the tip surface of the main body 20B of the gripper 20R. The palm sensor 26 is equipped with a high-resolution camera that identifies the type of luggage 100, and a MoPU (Motion Processing Unit) that identifies the position of the luggage 100.

As shown in FIG. 4(B), the suction pad 24 is composed of a rubber pad portion 24A that faces the baggage 100 when gripping it, and a nipple 24B that forms an air flow path to suck in air from the sealed space formed by the close contact between the pad portion 24A and the baggage 100.

In other words, the suction pad 24 in this embodiment has an air suction structure, and has suction power by sucking air from the sealed space through a hole 24C provided in the nipple 24B to create a vacuum (including a near vacuum). Note that the suction pad 24 is not limited to an air suction structure, and may have a structure in which suction is achieved by simply changing the volume of the sealed space by deforming the pad portion 24A.

Here, the size of the suction pad 24 varies depending on the mounting position (due to the difference in the diameter dimension of the pad portion 24A).

Broadly speaking, the suction pads 24 are classified into large diameter size and small diameter size, and are composed of suction pads 24X of relatively small diameter size (smaller than the large diameter size) and suction pads 24Y of relatively large diameter size (larger than the small diameter size). In the following, when it is not necessary to distinguish between sizes and they are referred to collectively, they may be simply referred to as suction pads 24.

A small-diameter suction pad 24X is attached to the finger portion 22A of the gripping portion 20L, and a large-diameter suction pad 24Y is attached to the other finger portions 22B, 22C, and the palm side 20A.

In addition, a large-diameter suction pad 24Y is attached to the main body of the gripper 20R.

Although the suction pads 24X and 24Y are provided in two sizes, they may be provided in three or more sizes.

The high-resolution camera that constitutes the palm sensor 26 of this embodiment identifies the photographed luggage 100 based on the captured image information, and determines whether it is a care product such as shampoo, conditioner, cosmetics, or toothpaste, or whether it is food such as instant ramen or a bag of sweets.

In other words, the high-resolution camera serves to obtain information to identify the type of luggage 100 (shape, size, hardness, etc.).

Meanwhile, the MoPU, which constitutes the palm sensor 26 of this embodiment together with the high-resolution camera, outputs, from an image of the luggage 100 captured at a frame rate of 1000 frames/second or more, motion information indicating the motion of the captured luggage 100 (in this case, the relative motion between the arms 5, 6) at a frame rate of, for example, 1000 frames/second or more. Note that when detecting a moving luggage 100, the frame rate may be increased, and when detecting a fixed object (luggage 100 that does not move), the frame rate may be decreased.

The MoPU outputs, as motion information, vector information on the movement of a point indicating the location of the luggage 100 along a specified coordinate axis. In other words, the motion information output from the MoPU does not include information necessary to identify what the photographed luggage 100 is (the care product or food) and only includes information indicating the movement (direction and speed of movement) of the center point (or center of gravity) of the luggage 100 on the coordinate axes (x-axis, y-axis, z-axis).

In other words, the trajectory of the gripping portion 20 as it approaches the luggage 100 can be accurately guided.

Information output from the palm sensor 26, which includes a high-resolution camera and MoPU, is supplied to the information processing device 14.

The information processing device 14 uses information from a palm sensor 26 including a high-resolution camera and MoPU to pinpoint the position of the luggage 100 with high precision, calculates the degree of spread of the fingers 22A, 22B, 22C when gripping, the strength of the grip, and the suction force of the suction pad 24, and accurately controls the minute movements of the arms 5, 6 and the gripping part 20, making it possible to handle the picking of various luggage 100.

(Selection of suction pads 24X and 24Y)
In this embodiment, the suction pads 24X and 24Y are selected based on the weight information of the baggage 100.

In this embodiment, the luggage 100 to be picked can be classified as shown in Table 1 below.

Table 1 classifies the objects 100 to be picked according to three categories: shape, weight, and other condition.

The shape category can be set to Box, Not box, or Other. The weight category can be set to Over 5kg or Below 5kg. The state category can be set to Aligned, which is a linear shape that is primarily polygonal, such as a rectangle, or Not aligned, which is a nonlinear shape that is primarily a sphere or bag.

In the comparative example, an applicable robot is selected based on the attributes of the luggage 100 determined by combining the above items, and the picking work is performed. That is, the types of robots include finger robots, sucking robots in "large", "medium", and "small", and two-arm robots, and an applicable robot is selected from among these.

However, when selecting a different type of robot, for example, if a robot waiting nearby is of an inapplicable type, a new robot of an applicable type must be called in, making the picking process less efficient.

In this embodiment, a single humanoid robot 1 is provided with multiple different gripping parts (grip parts 20L, 20R) and is also fitted with suction pads of different sizes (suction pads 24X, 24Y). Based on the weight information of the luggage 100 that can be known in advance, a gripping function is selected from one or two arms 5, 6 (fingers 22A, 22B, 22C) attached to the single humanoid robot 1, and the luggage 100 is picked up by suction and/or gripping operations.

In addition, by having multiple grasping functions, one humanoid robot 1 can select a grasping action based on weight information, and can actually approach the luggage 100, identify the shape information (size) of the luggage 100 from the image information detected by the palm sensor 26, and if there is a possibility that grasping will be unstable (predetermined shape information (size) deviates from the acceptable range) with the selected arm parts 5, 6 (fingers 22A, 22B, 22C), the grasping form can be changed on-site (in a confrontational state).

In other words, in this embodiment, following the selection of a picking task based on weight information, the picking task is changed based on shape information (size) as necessary (using detection information from the palm sensor 26).

(Selection of picking operations based on weight information)
In this embodiment, depending on the weight information of the luggage 100 determined in Table 1, the object size is classified into five patterns (extra-large, large, medium, small, extra-small) as shown in Table 2, and a picking task corresponding to each of the five patterns is selected (suitable for: right hand, left hand palm, left 2 fingers, left 1 finger).

More specifically, the following selection control is performed:

(Selection Control 1) In the case of an extra-large size package 100, it is determined to be the heaviest package 100 and is picked using the right hand grip 20R.

(Selection Control 2) In the case of a large-sized package 100, the package is picked using the suction pad 24Y attached to the palm side 20A of the left hand grip 20L (four large-diameter suction pads 24Y are used).

(Selection Control 3) In the case of a medium-sized piece of luggage 100, the luggage is picked using two suction pads 24Y attached to the finger portions 22B and 22X of the left gripping portion 20L (two large-diameter suction pads 24Y are used).

(Selection Control 4) In the case of a small-sized package 100, the package is picked using the finger portion 22B of the left hand gripper 200L (using the large-diameter suction pad 24Y).

(Selection Control 5) For extra-small size luggage 100, pick it using the finger portion 22A of the gripper 20L of the left hand (using the small diameter suction pad 24X).

(Selection Control 6) Although not shown in Table 2, if the luggage 100 is suitable for gripping, the luggage 100 is grasped and lifted with the fingers 22A, 22B, and 22C of the gripping portion 20L of the left hand. This selection control 6 may be used in combination with the above selection controls 1 to 5, or may be executed alone.

(Changes in picking operations based on shape information (size))
Here, the selection controls 1 to 6 based on Tables 1 and 2 described above select the size of the suction pad 24 based on the weight information of the baggage 100.

On the other hand, for example, when determining the weight of a piece of luggage 100 for which extra-small size is selected based on selection control 5, if the piece of luggage 100 is a certain size or larger, the gripping state may become unstable depending on the balance of the piece of luggage 100 after it is gripped.

Therefore, in this embodiment, in addition to selection control 1 to selection control 6 based on Tables 1 and 2 above, change control of the picking work is also performed.

Change control is a control in which, before the humanoid robot 1 faces the luggage 100 and grasps it, the palm sensor 26 is used to photograph the actual appearance of the luggage 100 to obtain shape information (size), and based on a pre-stored suction pad size-shape tolerance table, it is determined whether the shape information (size) of the selected suction pad 24 is within the tolerance range, and if the shape information (size) is larger than a certain value and exceeds the tolerance range, the selected suction pad 24 is changed.

When carrying out this change control, the information processing device 14 of the humanoid robot 1 stores a suction pad size-shape tolerance table (see Table 3) in advance.

Here, when the humanoid robot 1 approaches the grasping site and faces the luggage 100, it captures an image of the luggage 100 with the palm sensor 26 and obtains shape information (size) from the captured image information.

The information processing device 14 determines whether the acquired shape information is within the allowable range for the selected suction pad size, and if it is outside the allowable range, changes the suction pad size.

Figure 6 is a schematic diagram of an example of a control system for a humanoid robot according to this embodiment. The control system 10 includes a sensor 12 mounted on the humanoid robot, a palm sensor 26 including a high-resolution camera and a MoPU, and an information processing device 14.

The sensor 12 sequentially acquires information indicative of at least the distance and angle between the luggage 100 around the humanoid robot 1 and the arms 5 and 6 on which the humanoid robot 1 is working. The sensor 12 may be a high-performance camera, a solid-state LiDAR, a multi-color laser coaxial displacement meter, or a variety of other sensors. Other examples of the sensor 12 include a vibration meter, a thermo camera, a hardness meter, a radar, a LiDAR, a high-pixel, telephoto, ultra-wide-angle, 360-degree, high-performance camera, vision recognition, fine sound, ultrasound, vibration, infrared, ultraviolet, electromagnetic waves, temperature, humidity, spot AI weather forecast, high-precision multi-channel GPS, low-altitude satellite information, or long-tail incident AI data.

In addition to the above information, the sensor 12 detects images, distance, vibration, heat, smell, color, sound, ultrasound, ultraviolet light, infrared light, etc. Other information detected by the sensor 12 includes the movement of the center of gravity of the humanoid robot 1, the material of the floor on which the humanoid robot 1 is placed, the outside air temperature, the outside air humidity, the up/down/side/diagonal inclination angle of the floor, the amount of moisture, etc. The sensor 12 performs these detections, for example, every nanosecond.

The palm sensor 26 (high-resolution camera and MoPU) is a sensor provided on the gripping portion 20 of the arm portions 5 and 6, and has a camera function for photographing the luggage 100 and a positioning function for identifying the position of the luggage 100, separate from the sensor 12.

When one MoPU 12 is used, it is possible to obtain vector information of the movement of a point indicating the location of luggage 100 along each of two coordinate axes (x-axis and y-axis) in a three-dimensional orthogonal coordinate system. Using the principle of a stereo camera, two MoPUs 12 may be used to output vector information of the movement of a point indicating the location of luggage 100 along each of three coordinate axes (x-axis, y-axis, z-axis) in a three-dimensional orthogonal coordinate system. The z-axis is the axis along the depth direction (vehicle travel).

The information processing device 14 includes an information acquisition unit 140, a control unit 142, and an information storage unit 144.

The information acquisition unit 140 acquires information about the luggage 100 detected by the sensor 12 and the palm sensor 26 (high-resolution camera and MoPU).

The control unit 142 uses the information acquired by the information acquisition unit 140 from the sensor 12 and AI (artificial intelligence) to control the rotational movement of the connecting unit 4, the movement of the arms 5 and 6, and the like.

The control unit 142 also uses the information acquired by the information acquisition unit 140 from the palm sensor 26 (high-resolution camera and MoPU) to determine the type (shape, size, hardness, etc.) and position of the luggage 100 in detail, and controls the palm side 20A to face the luggage 100 according to its external shape and position, so that the luggage 100 is attracted by the suction pad 24 and grasped with the three fingers 22A, 22B, and 22C (grasping control). Note that the type of luggage 100 may be identified based on the external shape information, and the gripping control may be selected (such as "suction" only, "grasping" only, or a combination of "suction" and "grasping").

For example, the control unit 142 executes the following processes as an overall operation.
(1) The connecting portion 4 is driven to tilt the upper body portion 2 forward or backward so that the luggage 100 on a shelf or the floor can be picked up.
(2) The arms 5, 6 and the gripping portion are driven so that the luggage 100 can be grasped.
(3) The upper body 2 is driven up and down relative to the legs 3 to match the height of the workbench on the production line.
(4) Maintain balance to prevent the humanoid robot 1 from falling over.
(5) The drive of the wheels 7, 8 is controlled so that the humanoid robot 1 can push a cart or the like.

For example, when picking up luggage 100 that is on the floor, the information processing device 14 repeatedly executes the flowchart shown in FIG. 7.

In step S100, the information acquisition unit 140 acquires information about the luggage 100 detected by the sensor 12.

In step S102, the control unit 142 uses the information about the luggage 100 acquired in step S100 and AI to control the connecting unit 4 and the arms 5 and 6 to pick up the luggage 100 that is lying on the floor.

In step S104, the control unit 142 moves the picked-up luggage 100 to a specified position.

According to this embodiment, the humanoid robot 1 comprises an upper body 2, legs 3, and a connecting part 4 that connects the upper body 2 and legs 3 in a freely rotatable manner. The rotation of the connecting part 4 is controlled based on information acquired by a sensor 12. This makes it possible to determine the distance and angle between the humanoid robot 1 and the luggage 100, thereby enabling the robot to perform an action such as picking up luggage 100 that is lying on the floor.

In addition, the connecting part 4 allows the distance between the upper body part 2 and the legs 3 to be changed, so that the vertical position of the upper body part 2 relative to the legs 3 can be adjusted to match the height of the workbench on the production line.

The legs 3 also have a balancing function to prevent the humanoid robot 1 from falling over when the upper body 2 leans forward or backward relative to the legs 3. This makes it possible to prevent the humanoid robot 1 from falling over when performing work such as pushing or pulling a load 100 on a production line. This makes it possible to prevent breakdown of the humanoid robot 1 due to falling over, or injury to people around the humanoid robot 1.

(Grabbing control of luggage 100)
Fig. 8 is a flow chart showing the procedure of gripping control when gripping a package 100 with the gripper 20, which is an operation necessary for warehousing and receiving operations and is linked to the overall operation of the humanoid robot 1 in Fig. 5. The gripping control is basically executed by the information processing device 14 mounted on each humanoid robot 1, but the gripping control of the humanoid robots 1 present in the storage base 50 shown in Fig. 1 may be centrally controlled by a storage base management server 56.

In step 150, it is determined whether or not an instruction to grasp the luggage 100 has been given. If a positive determination is made, the process proceeds to step 152, where the humanoid robot 1 is moved (e.g., the arms 5 and 6 are operated) so that the palm side 20A faces the target luggage 100, and the process proceeds to step 154.

In the next step 154, an operation for gripping the luggage 100 is selected. For example, a selection is made from "suction" only, "grabbing" only, or a combination of "suction" and "grabbing". Then, the process proceeds to step 156, where a selection process for the gripping unit 20 and the suction pad 24 is executed according to the attributes of the luggage 100 (see FIG. 9 for details), and the process proceeds to step 158.

In step 158, the palm side 20A is turned toward the palm sensor 26, and information about the luggage 100 is detected from the information detected by the palm sensor 26 (such as the image captured by the high-resolution camera and the position information detected by the MoPU).

In the next step 160, the information detected by the palm sensor 26 is analyzed to obtain a detailed understanding of the type (shape, size, hardness, etc.) and location of the luggage 100, and the process proceeds to step 162.

In step 162, the suction pad size-shape tolerance table (see Table 3) previously stored in the information processing device 14 is read, and then the process proceeds to step 164 to determine whether the size of the selected suction pad 24 is within the tolerance range for the luggage 100 to be grasped.

If the result of step 164 is negative, the process proceeds to step 166, where the size of the suction pad 24 is changed based on Table 3, and the process proceeds to step 168.

Also, if the answer is yes in step 164, no changes are necessary, so proceed to step 168.

In step 168, grasping of the luggage 100 is performed ("suction" only, "grab" only, "suction" and "grab").

In the next step 170, it is determined whether the gripping of the luggage 100 was successful, and if the determination is affirmative, the gripped luggage 100 is carried to a predetermined location, and the process proceeds to step 150 to wait for an instruction to grip the next luggage 100.

If the result of step 170 is negative, the process proceeds to step 172, where error processing (e.g., retry or cancel) is performed, and the process returns to step 150.

(Grip and suction pad selection process)
FIG. 9 is a control flow chart showing the gripper/suction pad selection processing routine executed in step 156 of FIG.

As shown in FIG. 9, in step 170, it is determined whether the luggage 100 is extra-large. If the determination in step 170 is positive (extra-large size is confirmed), the process proceeds to step 172, where the gripper 20R on the right side is selected, and the four suction pads 24 attached to the gripper 200R are selected, and the process proceeds to step 190.

If the result of step 170 is negative, the process proceeds to step 174, where the left-hand grip 20L is selected, and the process proceeds to step 176.

In step 176, it is determined whether the baggage 100 is large. If the determination in step 176 is positive (determined to be large), the process proceeds to step 178, where the four suction pads 24 on the palm side 20A of the gripping portion 20L are selected, and the process proceeds to step 190.

Also, if the result of step 176 is negative, the process proceeds to step 180.

In step 180, it is determined whether the luggage 100 is medium-sized. If the determination in step 180 is affirmative (medium size confirmed), the process proceeds to step 182, where the two large-diameter suction pads 24Y attached to the fingers 22B and 22C of the gripper 200L are selected, and the process proceeds to step 190.

Also, if the result of step 180 is negative, the process proceeds to step 184.

In step 184, it is determined whether the luggage 100 is small or not. If the determination in step 184 is affirmative (small size confirmed), the process proceeds to step 186, where one of the large-diameter suction pads 24Y attached to the finger portion 22B or finger portion 22C of the gripper 200L is selected, and the process proceeds to step 190.

Also, if a negative judgment is made in step 184 (extra small size is confirmed), the process proceeds to step 188, where the small diameter suction pad 24X attached to the finger portion 22A of the gripper 20L is selected, and the process proceeds to step 190.

In step 190, it is determined whether or not grasping with the fingers 22A, 22B, and 22C is necessary. If the determination is affirmative, the process proceeds to step 192, where a grasping operation with the three fingers 22A, 22B, and 22C of the gripping portion 20L on the left hand is selected, and the routine ends. If the determination is negative in step 190, the routine ends.

As described above, according to this embodiment, the gripping portion 20 is provided with three fingers 22A, 22B, and 22C, and a plurality of suction pads 24 are attached to the palm side 20A of the gripping portion 20 and to the fingers 22A, 22B, and 22C. The suction pads 24, for example, have an air suction structure, and can suction the luggage 100, and the luggage 100 can be grasped by bending the fingers 22A, 22B, and 22C.

Furthermore, by classifying the luggage 100 based on its attributes (shape, weight, etc.) and selecting the gripper 20 and the suction pad 24 based on the classification results, it becomes possible for a single humanoid robot 1 to pick various types of luggage 100. In other words, there is no need to select and dispatch a different robot for each attribute of the luggage 100, and the efficiency of the picking work can be improved.

A palm sensor 26 including a high-resolution camera and MoPU is attached to the palm side 20A, and the gripper 20 with the above structure is attached to the arms 5 and 6 of the humanoid robot 1, so that objects can be picked up reliably by the suction surface, and the baggage 100 can be carried without dropping from the gripper 20 even if the humanoid robot 1 moves quickly.

In addition, the palm sensor 26 (high-resolution camera and MoPU) is mounted on the palm side 20A, so the baggage 100 can be captured with high accuracy, and tasks that require minute movements can also be handled.

Furthermore, very soft and fragile items can be grasped by the movement of the fingers 22A, 22B, and 22C without using the suction pad 24, and damage to the soft cargo 100 can be prevented by adjusting the gripping force.

(Control system for outgoing and incoming work on a storage basis)
FIG. 10 is a functional block diagram of a control system for executing a warehousing operation and a warehousing operation, which is executed in cooperation with the storage base management server 56 and each information processing device 14 of the humanoid robot 1.

The storage base management server 56 includes a work status management unit 62. The work status management unit 62 is connected to a business schedule database 64, a baggage attribute information database 72, and a communication I/F 66.

The communication I/F 66 communicates between the information processing device 14 of the humanoid robot 1 and the control device 54A of the transport device 54, and exchanges information.

The work status management unit 62 sequentially acquires information related to the work schedule from the work schedule database 64, and also acquires work progress information of the control objects (information processing device 14 of the humanoid robot 1, control device 54A of the transport device 54) performing work on the storage base 50 via the communication I/F 66, thereby managing the work schedule, including instructions for allocating the next work.

The communication I/F 66 is connected to the humanoid robot work status information acquisition unit 68, and acquires information regarding the work status when the humanoid robot 1 is working (e.g., the position of the picking station 58, the identification of the shelf 52 located in the opening 58A, whether the left or right gripper 20 is being used, etc.).

In other words, the humanoid robot work status information acquisition unit 68 acquires detailed information about specific tasks during outbound or inbound work, which is not important for allocating a regular work schedule.

The humanoid robot work information acquisition unit 68 is connected to the work status management unit 62.

The work status management unit 62 comprehensively analyzes the work status information from the humanoid robot work information acquisition unit 68, the work schedule information from the work schedule database 64, and the baggage attribute information from the baggage attribute information database 72, and compares the current progress with the schedule to provide overall management.

The work status management unit 62 normally assigns the next task after receiving a task completion notification from the humanoid robot 1 currently working, but in some cases, it assigns a task to the humanoid robot 1 currently working.

More specifically, assuming a situation in which a specific shelf 52 is placed at the frontage 58A of a specific picking station 58, and a picking operation is being performed by grasping a package 100 from the storage space 52A of the shelf 52, if the package 100 can be picked from the storage space of the same specific shelf 52, it is possible to instruct the system to continue operations in the current position (without returning to a home position or the like upon completion of operations).

(Normal work assignment instruction flow)
FIG. 11 is a flowchart showing a main routine of normal work instruction control based on a work schedule by the storage base management server 56, which is executed in the storage base 50 (see FIG. 1).

In step 200, the work schedule is read, and then the process proceeds to step 202, where the allocation process is performed. The allocation process in step 202 can be performed using a worksheet or the like that indicates which shelf 52 is to be transported to which picking station 58 by which transport device 54, and which humanoid robot 1 is to perform the outbound or inbound work.

In the next step 204, the transport device 54 is instructed to transport the shelf 52, and then in step 206, the humanoid robot 1 is instructed to perform a shipping or receiving operation, and the process proceeds to step 208.

In step 208, it is determined whether or not a work completion notification has been received. If the determination is affirmative, the process proceeds to step 210, where the work instruction object is confirmed, and then to step 212, where the work progress log for each work instruction object is updated, and the process proceeds to step 214.

Also, if there is no notification of the end of the business in step 208, the process proceeds to step 214 to continue with other business operations.

In other words, by monitoring the progress of each task as it is assigned according to the order set in the task schedule, it is possible to improve the efficiency of tasks performed by a limited number of transport devices 54 and a limited number of humanoid robots 1.

12 shows an example of a hardware configuration of a computer 1200 functioning as an information processing device 14. A program installed on the computer 1200 can cause the computer 1200 to function as one or more "parts" of the device according to the present embodiment, or to execute operations or one or more "parts" associated with the device according to the present embodiment, and/or to execute a process or steps of the process according to the present embodiment. Such a program can be executed by the CPU 1212 to cause the computer 1200 to execute specific operations associated with some or all of the blocks of the flowcharts and block diagrams described herein.

The computer 1200 according to this embodiment includes a CPU 1212, a RAM 1214, and a graphics controller 1216, which are connected to each other by a host controller 1210. The computer 1200 also includes input/output units such as a communication interface 1222, a storage device 1224, a DVD drive, and an IC card drive, which are connected to the host controller 1210 via an input/output controller 1220. The DVD drive may be a DVD-ROM drive, a DVD-RAM drive, or the like. The storage device 1224 may be a hard disk drive, a solid state drive, or the like. The computer 1200 also includes input/output units such as a ROM 1230 and a keyboard, which are connected to the input/output controller 1220 via an input/output chip 1240.

The CPU 1212 operates according to the programs stored in the ROM 1230 and the RAM 1214, thereby controlling each unit. The graphics controller 1216 acquires image data generated by the CPU 1212 into a frame buffer or the like provided in the RAM 1214 or into itself, and causes the image data to be displayed on the display device 1218.

The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD drive reads programs or data from a DVD-ROM or the like and provides them to the storage device 1224. The IC card drive reads programs and data from an IC card and/or writes programs and data to an IC card.

ROM 1230 stores therein a boot program, etc., executed by computer 1200 upon activation, and/or a program that depends on the hardware of computer 1200. I/O chip 1240 may also connect various I/O units to I/O controller 1220 via USB ports, parallel ports, serial ports, keyboard ports, mouse ports, etc.

The programs are provided by a computer-readable storage medium such as a DVD-ROM or an IC card. The programs are read from the computer-readable storage medium, installed in storage device 1224, RAM 1214, or ROM 1230, which are also examples of computer-readable storage media, and executed by CPU 1212. The information processing described in these programs is read by computer 1200, and brings about cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be constructed by realizing the operation or processing of information according to the use of computer 1200.

For example, when communication is performed between computer 1200 and an external device, CPU 1212 may execute a communication program loaded into RAM 1214 and instruct communication interface 1222 to perform communication processing based on the processing described in the communication program. Under the control of CPU 1212, communication interface 1222 reads transmission data stored in a transmission buffer area provided in RAM 1214, storage device 1224, a DVD-ROM, or a recording medium such as an IC card, and transmits the read transmission data to the network, or writes received data received from the network to a reception buffer area or the like provided on the recording medium.

The CPU 1212 may also cause all or a necessary portion of a file or database stored in an external recording medium such as the storage device 1224, a DVD drive (DVD-ROM), an IC card, etc. to be read into the RAM 1214, and perform various types of processing on the data on the RAM 1214. The CPU 1212 may then write back the processed data to the external recording medium.

Various types of information, such as various types of programs, data, tables, and databases, may be stored in the recording medium and may undergo information processing. The CPU 1212 may execute various types of processing on the data read from the RAM 1214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search/replacement, etc., as described throughout the present invention and specified by the instruction sequence of the program, and write back the results to the RAM 1214. The CPU 1212 may also search for information in a file, database, etc. in the recording medium. For example, when a plurality of entries, each having an attribute value of a first attribute associated with an attribute value of a second attribute, are stored in the recording medium, the CPU 1212 may search for an entry whose attribute value of the first attribute matches a specified condition from among the plurality of entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

The above-described programs or software modules may be stored in a computer-readable storage medium on the computer 1200 or in the vicinity of the computer 1200. In addition, a recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, thereby providing the programs to the computer 1200 via the network.

The blocks in the flowcharts and block diagrams of the present embodiment may represent stages of a process in which an operation is performed or "parts" of a device responsible for performing the operation. Particular stages and "parts" may be implemented by dedicated circuitry, programmable circuitry provided with computer-readable instructions stored on a computer-readable storage medium, and/or a processor provided with computer-readable instructions stored on a computer-readable storage medium. Dedicated circuitry may include digital and/or analog hardware circuitry, and may include integrated circuits (ICs) and/or discrete circuits. Programmable circuitry may include reconfigurable hardware circuitry including AND, OR, XOR, NAND, NOR, and other logical operations, flip-flops, registers, and memory elements, such as, for example, field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), and the like.

A computer-readable storage medium may include any tangible device capable of storing instructions that are executed by a suitable device, such that a computer-readable storage medium having instructions stored thereon comprises an article of manufacture that includes instructions that can be executed to create means for performing the operations specified in the flowchart or block diagram. Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like. More specific examples of computer-readable storage media may include floppy disks, diskettes, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), electrically erasable programmable read-only memories (EEPROMs), static random access memories (SRAMs), compact disk read-only memories (CD-ROMs), digital versatile disks (DVDs), Blu-ray disks, memory sticks, integrated circuit cards, and the like.

The computer readable instructions may include either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk (registered trademark), JAVA (registered trademark), C++, etc., and conventional procedural programming languages such as the "C" programming language or similar programming languages.

The computer-readable instructions may be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, or a programmable circuit, either locally or over a local area network (LAN), a wide area network (WAN), such as the Internet, to cause the processor of the general-purpose computer, special-purpose computer, or other programmable data processing apparatus, or a programmable circuit, to execute the computer-readable instructions to generate means for performing the operations specified in the flowcharts or block diagrams. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

The present invention has been described above using an embodiment, but the technical scope of the present invention is not limited to the scope described in the above embodiment. It is clear to those skilled in the art that various modifications and improvements can be made to the above embodiment. It is clear from the claims that forms with such modifications or improvements can also be included in the technical scope of the present invention.

The order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before" or "prior to," and it should be noted that the processes may be performed in any order, unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is explained using "first," "next," etc. for convenience, it does not mean that it is necessary to perform the processes in this order.

The present invention has been described above using an embodiment, but the technical scope of the present invention is not limited to the scope described in the above embodiment. It is clear to those skilled in the art that various modifications and improvements can be made to the above embodiment. It is clear from the claims that forms with such modifications or improvements can also be included in the technical scope of the present invention.

The order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before" or "prior to," and it should be noted that the processes may be performed in any order, unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is explained using "first," "next," etc. for convenience, it does not mean that it is necessary to perform the processes in this order.

1 Humanoid robot, 2 Upper body, 3 Legs, 4 Connecting part, 5, 6 Arms, 7, 8 Wheels, 10 Control system, 12 Sensor, 14 Information processing device, 20 (20L, 20R) Grip part, 20A Palm side, 20B Main body, 22A, 22B, 22C Finger part, 24 (24X, 24Y) Suction pad, 24A Pad part, 24B Nipple, 26 Palm sensor, 100 Luggage, 50 Storage base, 52 Shelf, 52A Storage space, 54 Transport device, 56 Storage base management server, 58 Picking station, 58A Frontage, 58B Work space, 60 Box, 62 Work status management unit, 64 Business schedule database, 66 Communication I/F, 68 Humanoid robot work status information acquisition unit, 72 Luggage attribute information database, 1200 Computer, 1210 host controller, 1212 CPU, 1214 RAM, 1216 graphic controller, 1218 display device, 1220 input/output controller, 1222 communication interface, 1224 storage device, 1230 ROM, 1240 input/output chip

Claims (5)

A control system for a robot capable of grasping an object using a first grasping unit having a palm portion serving as a base for grasping the object and a plurality of fingers attached to the palm portion, and a second grasping unit having a holder capable of grasping and holding the object,
a plurality of types of suction pads selectively provided for the palm portion and the finger portions of the first gripping portion, the suction pads being capable of suctioning and gripping the object with different suction areas;
a sensor unit provided on the first gripping unit and the second gripping unit, the sensor unit detecting at least shape information of the object;
a selection unit that selects, as a function for gripping the object, one of a first function for gripping the object with the first gripping unit, a second function for gripping the object with the second gripping unit, and a third function for using both the first function and the second function, based on weight information of the object that has been registered in advance;
A control unit that controls an operation of gripping the object based on the function selected by the selection unit;
a determination unit that determines whether or not shape information of the object detected by the sensor unit during an approach operation to the object in a gripping operation by the control unit is within an allowable range of a function of gripping the object selected by the selection unit;
a change unit that changes a function of gripping the object when the determination unit determines that the object is outside the allowable range; and
A robot control system having the following: The robot control system of claim 1, wherein the determination unit makes a determination based on a table in which the function of grasping the object is associated with an acceptable range of shape information of the object. The robot control system of claim 1, wherein the sensor unit includes a camera that captures an image of the object and identifies the type of the object, and a motion processing unit that identifies the position of the object. the first gripping portion has three fingers, a suction pad having a smaller diameter than the other suction pads is attached to a tip of the first finger, and suction pads having a larger diameter than the other suction pads are attached to tip portions of the second finger and the third finger,
The control unit is
classifying the objects into weight classification sizes of extra large, large, medium, small, and extra small based on the weight information;
When the object is the extra-large size, the object is gripped using a holder provided in the second gripping portion,
When the object is a large size, the object is grasped using a suction pad provided on the palm portion,
When the object is of a medium size, the object is gripped using the large-diameter suction pads provided at the tip portions of the second finger portion and the third finger portion;
When the object is small, the object is gripped using the large-diameter suction pad provided at the tip of the second finger portion or the third finger portion;
When the object is the extra-small size, the object is selected to be grasped using the small-diameter suction pad provided at the tip of the first finger portion.
The control system for the robot according to claim 1. A robot control program that causes a computer to operate as a selection unit, a control unit, a determination unit, and a change unit as described in any one of claims 1 to 4.