JP2019042853A - Image information processing apparatus, gripping system, and image information processing method - Google Patents

Abstract

To provide a technique for grasping which of plural planes is a plane contactable by a finger part of a hand mechanism, in an object that is a polyhedron.SOLUTION: On a prescribed plane of an object that is to become an imaging target using an imaging device, a prescribed position on each side forming the prescribed plane is preset as a determination point. Further, on the basis of image information acquired by the imaging device, it is determined for each of the determination points on the prescribed plane of the object whether the determination point is in contact with a peripheral object. Then, other planes neighboring the prescribed plane by interposing a side on which a determination point determined not to be in contact with a peripheral object is positioned, are set as finger contactable planes. However, other planes neighboring the prescribed plane by interposing a side on which a determination point determined to be in contact with a peripheral object is positioned, are set as finger non-contactable planes.SELECTED DRAWING: Figure 14

Description

  The present invention relates to a gripping system for gripping an object by a hand mechanism having a plurality of finger parts, and an image information processing apparatus and an image information processing method for processing image information including the object in the gripping system.

  Conventionally, a gripping system for gripping an object by a hand mechanism having a plurality of fingers has been developed. As a gripping system, one having an imaging device that captures an image including an object is known. In a gripping system including such an imaging device, an object is recognized based on image information acquired by the imaging device. Further, the hand mechanism is controlled based on the image information in order to hold the recognized object.

  For example, Patent Document 1 discloses a picking system (gripping system) including a monocular camera that captures a plurality of box-shaped workpieces from above and acquires an entire image. The picking system described in Patent Document 1 includes a distance sensor that measures a three-dimensional shape of a plurality of box-shaped workpieces in addition to a monocular camera. Then, an edge portion of the box-shaped workpiece is detected from the entire image acquired by the monocular camera, and a point group of measurement points by the distance sensor corresponding to the edge portion is extracted with the detected edge portion as a boundary. Furthermore, based on the extracted point group, the position and posture of each box-shaped workpiece are recognized from the three-dimensional shape measured by the distance sensor. After that, the position and posture of the box-shaped workpiece to be picked are output.

Japanese Patent No. 5429614

  When an object is to be gripped by the hand mechanism, it is necessary to bring the finger part of the hand mechanism into contact with a position on the surface of the object where the object can be sandwiched by the finger part of the hand mechanism. . However, when the target object is a polyhedron, depending on the mounting state of the target object, there may be a plane on which it is difficult to contact the finger part of the hand mechanism. That is, when an object is placed in contact with a peripheral object (for example, another object or a container that contains the object) that exists in the vicinity of the object, The finger part of the hand mechanism cannot be brought into contact with the plane of the object that is in contact with the surrounding object. Therefore, when the object is gripped by the hand mechanism, it is necessary to grasp which plane of the object can be touched by the finger portion of the hand mechanism.

  The present invention has been made in view of the above problems, and provides a technique capable of grasping which plane is a plane that can be touched by a finger portion of a hand mechanism in an object that is a polyhedron. The purpose is to do.

An image information processing apparatus according to the present invention is a gripping system that grips an object by a hand mechanism having a plurality of fingers, and includes an imaging device that captures an image including at least one object. In an image information processing apparatus that is applied and processes image information acquired by the imaging device, the target is a polyhedron, and the predetermined target plane is a target of imaging by the imaging device on the target. A predetermined position on each side forming the plane of the object is preset as a determination point, and the determination is made for each determination point of the predetermined plane of the object based on the image information acquired by the imaging device. A determination unit that determines whether or not the point is in contact with a peripheral object; and the determination unit determines that the point is not in contact with the peripheral object. In addition, another plane adjacent to the predetermined plane across the side where the determination point is located is set as a finger contactable surface that can be contacted by the finger portion of the hand mechanism, and the determination unit sets the periphery. A finger contact impossible surface that cannot be contacted by the finger portion of the hand mechanism on another plane adjacent to the predetermined plane across the side where the determination point determined to be in contact with an object is located And a setting unit for setting as

  ADVANTAGE OF THE INVENTION According to this invention, in the target object which is a polyhedron, it can grasp | ascertain which plane is a plane which can contact the finger | toe part of a hand mechanism.

It is a figure which shows schematic structure of the holding | grip system which concerns on an Example. It is a figure which shows schematic structure of the robot arm which concerns on an Example. It is a perspective view of the hand mechanism which concerns on an Example. It is a top view of the hand mechanism which concerns on an Example. It is a side view of the finger | toe part of the hand mechanism which concerns on an Example. It is the figure which looked at the front-end | tip part side of the finger | toe part of the hand mechanism which concerns on an Example from the direction of the arrow A of FIG. It is a figure which shows the internal structure of the connection part vicinity of the finger part in a base part, and the internal structure of the base end part in a finger part, and a 2nd joint part of the hand mechanism which concerns on an Example. It is a figure which shows the internal structure of the 1st joint part in a finger part, and the 2nd finger link part of the hand mechanism which concerns on an Example. It is a figure which shows the movable range of the 2nd joint part in the finger | toe part of the hand mechanism which concerns on an Example. It is a figure which shows the movable range of the 1st joint part in the finger | toe part of the hand mechanism which concerns on an Example. It is a figure which shows arrangement | positioning of the pressure sensor in the 1st link part of the finger | toe part of the hand mechanism which concerns on an Example. It is a block diagram which shows each function part contained in the arm control apparatus which concerns on an Example, a hand control apparatus, and an image information processing apparatus. It is a figure which shows an example of the shape of the target object, and the position of the determination point in this target object based on an Example. It is a 1st figure which shows the specific example of the state of each determination point of the upper surface of the said target object in image information when the target object is mounted in the state which contacted the surrounding object based on an Example. It is a 2nd figure which shows the specific example of the state of each determination point of the upper surface of the said target object in image information when the target object is mounted in the state which contacted the surrounding object based on an Example. It is a 3rd figure which shows the specific example of the state of each determination point of the upper surface of the said target object in image information when the target object is mounted in the state which contacted the surrounding object based on an Example. FIG. 10 is a fourth diagram illustrating a specific example of the state of each determination point on the upper surface of the target object in the image information when the target object is placed in contact with a peripheral object according to the example. It is a 5th figure which shows the specific example of the state of each determination point of the upper surface of the said target object in image information when the target object is mounted in the state which contacted the surrounding object based on an Example. 3 is a flowchart illustrating a flow of image information processing performed in the image information processing apparatus according to the first embodiment. FIG. 6 is a block diagram illustrating functional units included in an image information processing apparatus according to a second embodiment. 10 is a flowchart illustrating a flow of image information processing performed in the image information processing apparatus according to the second embodiment. FIG. 10 is a block diagram illustrating functional units included in an image information processing apparatus according to a third embodiment. FIG. 10 is a diagram illustrating a hand mechanism and a state of the target object when the target object is gripped by direct gripping according to a third embodiment. FIG. 10 is a first diagram illustrating a hand mechanism and a state of the target object when the target object is gripped by tilting gripping according to the third embodiment. It is a 2nd figure which shows the hand mechanism when the target object is gripped by tilting gripping, and the state of the target object according to the third embodiment. FIG. 10 is a third diagram illustrating a hand mechanism and a state of the target object when the target object is gripped by tilting gripping according to the third embodiment. 12 is a flowchart illustrating a flow of image information processing performed in the image information processing apparatus according to the third embodiment.

  Specific embodiments of the present invention will be described below with reference to the drawings. The dimensions, materials, shapes, relative arrangements, and the like of the components described in the present embodiment are not intended to limit the technical scope of the invention to those unless otherwise specified.

  The gripping system according to the present invention includes an imaging device that captures an image including an object. Then, the image information processing apparatus according to the present invention processes the image information acquired by the imaging apparatus in order to grasp which plane in the object is a plane that can be touched by the finger part of the hand mechanism.

  More specifically, in the present invention, when an object that is a polyhedron is imaged by the imaging device, a predetermined plane (that is, a plane facing the imaging device) of the object is an object to be imaged. It becomes. And in the predetermined plane of this object, the predetermined position on each side which forms the predetermined plane is preset as a determination point. According to this, the image information acquired by the imaging device includes the state of each determination point on a predetermined plane of the object. Therefore, in the image information processing apparatus according to the present invention, for each determination point on a predetermined plane of the target object, whether or not the determination point is in contact with a surrounding object based on the image information acquired by the imaging apparatus. Determined by the determination unit.

  Here, in the image information, when a certain determination point of a predetermined plane in the target object is in contact with a peripheral object, another plane adjacent to the predetermined plane across the side where the determination point is located (that is, The other plane in contact with the predetermined plane at the side where the determination point is located is considered to be in contact with the peripheral object. On the other hand, in the image information, when a determination point having a predetermined plane in the target object is not in contact with a surrounding object, the other plane adjacent to the predetermined plane is around the side where the determination point is located. It is considered that the object is not in contact. And the finger part of the hand mechanism cannot be brought into contact with the plane in contact with the peripheral object in the object as it is. On the other hand, the finger part of the hand mechanism can be brought into contact with a plane that is not in contact with the surrounding object in the object.

Therefore, in the image information processing apparatus according to the present invention, the setting unit sets another plane adjacent to the predetermined plane across the side where the determination point determined to be not in contact with the surrounding object by the determination unit is located. Then, it is set as a finger contactable surface that can be contacted by the finger part of the hand mechanism. On the other hand, another plane adjacent to the predetermined plane across the side where the determination point determined to be in contact with the surrounding object by the determination unit is not contacted by the finger of the hand mechanism by the setting unit. Set as possible finger contact impossible surface. According to this, it is possible to grasp which plane is a plane in which the finger part of the hand mechanism can be contacted in the placed object.

<Example 1>
Specific embodiments of the present invention will be described below with reference to the drawings. The dimensions, materials, shapes, relative arrangements, and the like of the components described in the present embodiment are not intended to limit the technical scope of the invention to those unless otherwise specified.

  Here, the case where the present invention is applied to a gripping system including a robot arm will be described. FIG. 1 is a diagram illustrating a schematic configuration of a gripping system according to the present embodiment. The gripping system 100 includes a robot arm 1 and an imaging device 8. The imaging device 8 is a camera, for example, and is installed above the object 10. Then, the imaging device 8 captures an image including the object 10 and its surroundings from above the object 10. The robot arm 1 also includes a hand mechanism 2 for holding the object 10, an arm mechanism 3, and a pedestal portion 4. Here, the detailed structure of the arm mechanism 3, the hand mechanism 2, and the base part 4 in the robot arm 1 will be described.

(Arm mechanism)
FIG. 2 is a diagram showing a schematic configuration of the robot arm 1. A hand mechanism 2 is attached to one end of the arm mechanism 3. The other end of the arm mechanism 3 is attached to the pedestal portion 4. The arm mechanism 3 includes a first arm link portion 31, a second arm link portion 32, a third arm link portion 33, a fourth arm link portion 34, a fifth arm link portion 35, and a connection member 36. The base portion 20 of the hand mechanism 2 is connected to a first joint portion 30 a formed on one end side of the first arm link portion 31 of the arm mechanism 3. The first joint portion 30 a is provided with a motor (not shown) for rotating the hand mechanism 2 around the axis of the first arm link portion 31 with respect to the first arm link portion 31. The other end side of the first arm link portion 31 is connected to one end side of the second arm link portion 32 at the second joint portion 30b. The first arm link part 31 and the second arm link part 32 are connected so that their central axes intersect perpendicularly. A motor for rotating the first arm link part 31 around the axis of the second arm link part 32 around the other end side with respect to the second arm link part 32 is provided in the second joint part 30b. (Not shown) is provided. Further, the other end side of the second arm link portion 32 is connected to one end side of the third arm link portion 33 at the third joint portion 30c. The third joint part 30 c is provided with a motor (not shown) for rotating the second arm link part 32 relative to the third arm link part 33.

Similarly, the other end side of the third arm link portion 33 is connected to one end side of the fourth arm link portion 34 at the fourth joint portion 30d. Further, the other end side of the fourth arm link portion 34 is connected to the fifth arm link portion 35 by a fifth joint portion 30e. The fourth joint portion 30d is provided with a motor (not shown) for rotating the third arm link portion 33 relative to the fourth arm link portion 34. Further, the fifth joint portion 30e is provided with a motor (not shown) for rotating the fourth arm link portion 34 relative to the fifth arm link portion 35. Further, the fifth arm link portion 35 is connected to a connection member 36 disposed vertically from the pedestal portion 4 by a sixth joint portion 30f. The fifth arm link portion 35 and the connection member 36 are connected so that their central axes are coaxial. The sixth joint portion 30f is provided with a motor (not shown) for rotating the fifth arm link portion 35 around the axes of the fifth arm link portion 35 and the connection member 36. With the arm mechanism 3 having such a configuration, for example, the arm mechanism 3 can be a mechanism having six degrees of freedom.

(Hand mechanism)
Next, the configuration of the hand mechanism 2 will be described with reference to FIGS. FIG. 3 is a perspective view of the hand mechanism 2. FIG. 4 is a top view of the hand mechanism 2. In FIG. 4, the arrows indicate the rotation movable ranges of the finger portions 21. The hand mechanism 2 includes a base portion 20 connected to the arm mechanism 3 and four finger portions 21 provided on the base portion 20. As shown in FIGS. 3 and 4, in the hand mechanism 2, the four fingers 21 on the base 20 are centered on the axis in the longitudinal direction of the hand mechanism 2 (the direction perpendicular to the paper surface in FIG. 4). Are arranged at equiangular intervals (ie, 90 deg intervals). The four finger portions 21 all have the same structure and the same length. However, the operation of each finger 21 is controlled independently.

  5 to 11 are diagrams for explaining the configuration of the finger portion 21 of the hand mechanism 2 and its driving mechanism. FIG. 5 is a side view of the finger portion 21. In FIG. 5, the base portion 20 is shown in a transparent state, and a partial internal structure of the finger portion 21 located inside the base portion 20 is also shown. FIG. 6 is a view of the distal end side of the finger portion 21 as viewed from the direction of arrow A in FIG. 5 and 6, a part of a second finger link part 212 of the finger part 21 to be described later is shown in a transparent state, and the internal structure of the second finger link part 212 is also shown. Yes.

  As shown in FIGS. 3 and 5, each finger portion 21 has a first finger link portion 211, a second finger link portion 212, and a proximal end portion 213. The base end portion 213 of the finger portion 21 is connected to the base portion 20. Here, the base end portion 213 is connected to the base portion 20 so as to be rotatable about the longitudinal axis of the finger portion 21 (direction perpendicular to the paper surface in FIG. 4) with respect to the base portion 20, as indicated by an arrow in FIG. Yes. In the finger part 21, one end of the second finger link part 212 is connected to the base end part 213. A second joint portion 23 is formed at a connection portion between the second finger link portion 212 and the base end portion 213.

  Here, based on FIG. 7, the drive mechanism of the base end part 213 and the drive mechanism of the 2nd joint part 23 are demonstrated. FIG. 7 is a view showing the internal structure of the base portion 20 in the vicinity of the connecting portion of the finger portion 21 and the internal structures of the base end portion 213 and the second joint portion 23 in the finger portion 21. As shown in FIG. 7, a gear 65, a gear 66, a second motor 52, and a third motor 53 are provided inside the base portion 20. The gear 65 is a gear for rotating the entire finger portion 21, and is connected to the rotation shaft of the base end portion 213. The gear 66 is connected to the rotation shaft of the third motor 53. The gear 65 and the gear 66 are engaged with each other. With such a configuration, when the third motor 53 rotates, the rotational force is transmitted to the rotation shaft of the base end portion 213 via the two gears 65 and 66. As a result, the base end portion 213 is rotationally driven, and accordingly, the entire finger portion 21 is rotationally driven within a range indicated by an arrow in FIG.

A worm wheel 63 and a worm 64 meshing with the worm wheel 63 are provided inside the second joint portion 23. The worm wheel 63 is connected to the rotation shaft of the second finger link part 212 in the second joint part 23. A worm 64 is connected to the rotating shaft of the second motor 52 provided in the base portion 20. With such a configuration, when the second motor 52 is rotationally driven, the rotational force is transmitted to the rotation shaft of the second finger link portion 212 by the worm 64 and the worm wheel 63. As a result, the second finger link part 212 is rotationally driven relative to the base end part 213. Here, FIG. 8 is a diagram illustrating a movable range of the second joint portion 23 in the finger portion 21 realized by the driving force of the second motor 52. As shown in FIG. 8, the second joint portion 23 is formed to be able to bend and extend. The driving force by the second motor 52 and the driving force by the third motor 53 are configured to be transmitted independently to the operation target.

  As shown in FIGS. 5 and 6, in the finger portion 21, one end of the first finger link portion 211 is connected to the other end of the second finger link portion 212. A first joint portion 22 is formed at a connection portion between the first finger link portion 211 and the second finger link portion 212. Here, the drive mechanism of the 1st joint part 22 is demonstrated based on FIG. FIG. 9 is a diagram illustrating an internal structure of the first joint portion 22 and the second finger link portion 212 in the finger portion 21. Two bevel gears 61 and 62 meshing with each other are provided inside the first joint portion 22. One bevel gear 61 is connected to the rotation shaft of the first finger link portion 211 in the first joint portion 22. The other bevel gear 62 is connected to the rotation shaft of the first motor 51 provided inside the second finger link portion 212. With such a configuration, when the first motor 51 is rotationally driven, the rotational force is transmitted to the rotation shaft of the first finger link portion 211 by the two bevel gears 61 and 62. As a result, the first finger link portion 211 is rotationally driven relative to the second finger link portion 212. Here, FIG. 10 is a diagram illustrating a movable range of the first joint portion 22 in the finger portion 21 realized by the driving force of the first motor 51. As shown in FIG. 10, the first joint portion 22 is formed to be able to bend and extend.

  In addition, as shown in FIGS. 3, 5, 6, and 11, in this embodiment, a pressure-sensitive sensor 70 is provided on the distal end side of the first finger link portion 211 of the finger portion 21. The pressure-sensitive sensor 70 is a sensor that detects an external force (pressure) acting on the distal end portion of the first finger link portion 211. As shown in FIG. 5, the pressure-sensitive sensors 70 are provided on both surfaces of the wall surface 215 on the bending direction side and the wall surface 216 on the extension direction side of the first joint portion 22 in the first finger link portion 211.

(Pedestal)
Next, configurations of the arm control device 42, the hand control device 43, and the image information processing device 44 built in the pedestal unit 4 will be described with reference to FIG. The arm control device 42 is a control device for controlling the arm mechanism 3 of the robot arm 1. The hand control device 43 is a control device for controlling the hand mechanism 2 of the robot arm 1. The image information processing device 44 is a processing device for processing image information acquired by capturing an image including the object 10 by the imaging device 8. FIG. 12 is a block diagram illustrating functional units included in the arm control device 42, the hand control device 43, and the image information processing device 44.

  The arm control device 42 includes a plurality of drivers that generate drive signals for driving the motors provided at the joints of the arm mechanism 3 so that the drive signals from the drivers are supplied to the corresponding motors. Configured. The arm control device 42 includes a computer having an arithmetic processing device and a memory. The arm control device 42 includes an arm control unit 420 and a motor state quantity acquisition unit 421 as functional units. These functional units are formed by executing a predetermined control program in a computer included in the arm control device 42.

The arm control unit 420 supplies each driver of the arm mechanism 3 by supplying a drive signal from each driver based on target information acquired by a target information acquisition unit 430 described later, which is a functional unit of the hand control device 43. The motors provided in the joint portions 30a, 30b, 30c, 30d, 30e, and 30f are controlled. Then, the arm control unit 420 controls the motors to move the arm mechanism 3, thereby changing the position of the hand mechanism 2 to a desired position (position where the hand mechanism 2 can grip the object 10). Move to. In addition, the motors provided in the joint portions 30a, 30b, 30c, 30d, 30e, and 30f of the arm mechanism 3 detect state quantities (such as the rotational position and rotational speed of the rotation shaft of the motor) related to the respective rotational states. An encoder (not shown) is provided. Then, the state quantity of each motor detected by the encoder of each motor is input to the motor state quantity acquisition unit 421 of the arm control device 42. Then, the arm control unit 420 servo-controls each motor so that the hand mechanism 2 moves to a desired position based on the state quantity of each motor input to the motor state quantity acquisition unit 421.

  The hand control device 43 includes a plurality of drivers that generate drive signals for driving the motors provided in the hand mechanism 2 so that the drive signals from the drivers are supplied to the corresponding motors. Composed. The hand control device 43 includes a computer having an arithmetic processing device and a memory. And the hand control apparatus 43 has the target object information acquisition part 430, the hand control part 431, the motor state quantity acquisition part 432, and the sensor information acquisition part 433 as a function part. These functional units are formed by executing a predetermined control program in a computer included in the hand control device 43.

  The object information acquisition unit 430 acquires object information that is information related to an object to be gripped by the hand mechanism 2 from the image information processing apparatus 44. In addition, the hand control unit 431 supplies the driving signal from each driver based on the object information acquired by the object information acquiring unit 430, thereby driving each finger unit 21 of the hand mechanism 2. 51, each 2nd motor 52, and each 3rd motor 53 are controlled. Thereby, the target object 10 is gripped by the plurality of finger portions 21 of the hand mechanism 2. In addition, the first motor 51, the second motor 52, and the third motor 53 of the hand mechanism 2 detect state quantities (such as the rotational position and rotational speed of the rotation shaft of the motor) related to the respective rotational states. An encoder (not shown) is provided. Then, the state quantities of the motors 51, 52, 53 detected by the encoders of the motors 51, 52, 53 are input to the motor state quantity acquisition unit 432 of the hand control device 43. The hand control unit 431 then moves each finger unit 21 so that the plurality of finger units 21 hold the object based on the state quantities of the motors 51, 52, and 53 input to the motor state quantity acquisition unit 432. Servo-controls the motors 51, 52, and 53 in FIG.

  Furthermore, the hand control device 43 has a sensor information acquisition unit (not shown). A detection value of the pressure-sensitive sensor 70 provided in the first finger link unit 211 of each finger unit 21 of the hand mechanism 2 is input to the sensor information acquisition unit. And when the contact of each finger part 21 to the object is detected by each pressure-sensitive sensor 70, the hand control unit 431, based on the detection signal, each motor 51, 52, 53 in each finger part 21. Can also be controlled.

  The image information processing device 44 includes a computer having an arithmetic processing device and a memory. The image information processing apparatus 44 includes an image information acquisition unit 440, an object recognition unit 441, a determination unit 442, a setting unit 443, and a gripping surface determination unit 444 as functional units. These functional units are formed by executing a predetermined control program in a computer included in the image information processing apparatus 44. Further, the processing result of the image information by the image information processing device 44 is input to the object information acquisition unit 430 of the hand control unit 431 as the object information. The functions performed by the respective function units in the image information processing apparatus 44 will be described later.

(Image information processing)
Here, the image information processing performed in the image information processing apparatus 44 will be described with reference to FIGS. FIG. 13 is a diagram illustrating an example of the shape of the object 10 and the position of a determination point on the object to be described later. As shown in FIG. 13, the shape of the target object 10 is a rectangular parallelepiped. And in the holding | grip system 100 which concerns on a present Example, as above-mentioned, the image containing the target object 10 and its periphery is imaged with the imaging device 8 from the upper direction of this target object. Therefore, the upper surface S <b> 1 of the object 10 is an object to be imaged by the imaging device 8. And in the target object 10, each plane which adjoins this upper surface S1 across the upper surface S1 and its each side, ie, each side surface of the target object 10, is made into the side surfaces S2-S5.

  When the object 10 is to be gripped by the hand mechanism 2, it is necessary to bring the finger part 21 into contact with any one of the side surfaces S <b> 2 to S <b> 5 of the object 10 and pinch the object 10 by the finger part 21. However, when the target object 10 is placed in contact with a peripheral object such as a container or other target object, a hand is not placed on the side surface of the target object 10 that is in contact with the peripheral object. The finger part 21 of the mechanism 2 cannot be brought into contact. Therefore, when the object 10 is gripped by the hand mechanism 2, which side surface of the placed object 10 can be contacted by the finger portion 21 (hereinafter referred to as “finger contactable surface”). It is necessary to determine the side surface (hereinafter also referred to as “gripping surface”) on which the finger portion 21 is actually brought into contact for grasping. .

  Therefore, in this embodiment, the finger contactable surface of the object 10 is grasped and the grip surface is determined by image information processing in the image information processing device 44. Specifically, in this embodiment, the midpoint of each side forming the upper surface S1 to be imaged by the imaging device 8 in the object 10 and each vertex of the upper surface are preset as determination points. In FIG. 13, each determination point on the upper surface S <b> 1 of the object 10 is indicated by P <b> 1 to P <b> 8.

  As shown in FIG. 13, when the object 10 is placed in a state where it is not in contact with any surrounding objects, image information acquired by being imaged by the imaging device 8 (hereinafter simply “image information”). In other words, the determination points P1 to P8 on the upper surface S1 of the object 10 are not in contact with surrounding objects. In other words, in the image information, if each of the determination points P1 to P8 on the upper surface S1 of the target object 10 is not in contact with a peripheral object, each of the side surfaces S2 to S5 of the target object 10 is a peripheral object. Can be judged as not touching. In this case, all of the side surfaces S2 to S5 of the object 10 are set as finger-contactable surfaces.

  14 to 18 are specific examples of states of the determination points P1 to P8 on the upper surface S1 of the target object 10 in the image information when the target object 10 is placed in contact with a peripheral object. FIG. In FIG. 13 to FIG. 18, the determination points that are not in contact with the surrounding objects are marked with a circle, and the determination points that are in contact with the surrounding objects are marked with a cross.

  In FIG. 14, the object 10 is placed in a state where the side surface S <b> 5 of the object 10 is in contact with a wall (for example, the inner wall surface of the container). In this case, in the image information, the determination points P1, P8, and P7 set on the side sandwiched between the upper surface S1 and the side surface S5 of the object 10 are in contact with the wall. Then, the other determination points P2, P3, P4, P5, and P6 are not in contact with the surrounding objects.

  Moreover, in FIG. 15, the target object 10a and the target object 10b are mounted side by side in the state which mutually contacted. In FIG. 15, the side surface S5 of the object 10a and the side surface S3 of the object 10b are in contact with each other. In this case, in the image information, the determination point P1, P8, P7 set on the side sandwiched between the upper surface S1 and the side surface S5 of the object 10a is in contact with the object 10b. Then, the other determination points P2, P3, P4, P5, and P6 on the object 10a are not in contact with surrounding objects. On the other hand, for the object 10b, the determination points P3, P4, and P5 set on the side sandwiched between the upper surface S1 and the side surface S3 are in contact with the object 10a. Then, the other determination points P1, P2, P6, P7, and P8 on the object 10b are not in contact with surrounding objects.

Further, in FIG. 16, in addition to the object 10a and the object 10b shown in FIG. 15, the object 10c is placed side by side in contact with the object 10a. In FIG. 16, the side surface S3 of the object 10a and the side surface S5 of the object 10c are in contact. In this case, in the image information, for the object 10a, the determination points P1, P8, P7 are in contact with the object 10b, and the determination is set on the side sandwiched between the upper surface S1 and the side surface S3. Points P3, P4, and P5 are also in contact with the object 10c. Then, the other determination points P2 and P6 in the object 10a are not in contact with the surrounding objects. Moreover, about the target object 10c, the determination points P1, P8, and P7 set on the side sandwiched between the upper surface S1 and the side surface S5 are in contact with the target object 10a. Then, the other determination points P2, P3, P4, P5, and P6 on the target object 10c are not in contact with the surrounding objects.

  Moreover, in FIG. 17, in addition to the target object 10a, the target object 10b, and the target object 10c shown in FIG. 16, the target object 10d is placed in contact with the target object 10a. In FIG. 17, the side surface S4 of the object 10a and the side surface S2 of the object 10d are in contact. In this case, in the image information, for the object 10a, the determination points P1, P8, and P7 are in contact with the object 10b, and the determination points P3, P4, and P5 are in contact with the object 10c. Determination points P5, P6, and P7 set on the side sandwiched between the upper surface S1 and the side surface S4 are also in contact with the object 10d (the determination point P7 is in contact with both the object 10b and the object 10d). In addition, the determination point P5 is in contact with both the object 10c and the object 10d.) And only the other determination point P2 in the target object 10a will be in the state which is not in contact with the surrounding object. In addition, for the object 10d, the determination points P1, P2, and P3 set on the side sandwiched between the upper surface S1 and the side surface S2 are in contact with the object 10a. Then, the other determination points P4, P5, P6, P7, and P8 on the object 10d are not in contact with the surrounding objects.

  In FIG. 18, the wall is in contact with the side surface S5 of the object 10a instead of the object 10b in FIG. Further, the side surface S5 of the object 10d is also in contact with this wall. In this case, in the image information, for the object 10a, the determination points P3, P4, and P5 are in contact with the object 10c, and the determination points P5, P6, and P7 are in contact with the object 10d. Determination points P1, P8, and P7 set on the side sandwiched between the upper surface S1 and the side surface S5 are in contact with the wall. Further, for the object 10d, the determination points P1, P2, and P3 are in contact with the object 10a, and the determination points P1, P8, which are set on the side sandwiched between the upper surface S1 and the side surface S5, P7 comes into contact with the wall. Then, the other determination points P4, P5, and P6 on the object 10d are not in contact with the surrounding objects.

  As described above, when a certain side surface (the side surface indicated by hatching in FIGS. 14 to 18) of the target object 10 (10a, 10b, 10c, 10d) is in contact with a peripheral object, the target object 10 is displayed in the image information. The determination point on the side sandwiched between the upper surface and the side surface of the lens is in contact with the peripheral object. In other words, it can be determined that the side surface adjacent to the top surface S1 is in contact with the peripheral object across the side where the determination points in contact with the peripheral object are arranged in the image information. Then, the side surface of the object 10 that is in contact with the surrounding object is set as a side surface that cannot be contacted by the finger portion 21 of the hand mechanism 2 (hereinafter also referred to as “finger contact impossible surface”). be able to. Further, the side surface adjacent to the upper surface S1 across the side where the determination point that is not in contact with the surrounding object in the image information is located can be set as a finger contactable surface.

  Therefore, in the image information processing apparatus 44 according to the present embodiment, the determination unit 442 determines whether or not each determination point on the upper surface S1 of the target object 10 is in contact with the surrounding object in the image information. And based on the determination result by the determination part 442, each side surface of the target object 10 is set as a finger contact possible surface or a finger contact impossible surface.

(Image information processing flow)
Next, the flow of image information processing performed in the image information processing apparatus 44 will be described based on the flowchart shown in FIG. This flow of image information processing is realized by executing a predetermined processing program in the image information processing apparatus 44. In this flow, first, in S101, image information is acquired from the imaging apparatus 8 by the image information acquisition unit 440. In step S <b> 102, the object recognition unit 441 recognizes the object 10 from the image information acquired by the image information acquisition unit 440. Here, in the image information processing apparatus 44, the shape and dimensions of the object 10 are input in advance by the user. The target object recognition unit 441 recognizes the target object 10 by extracting the target object 10 from the image information based on the previously input information. Further, if the plurality of objects 10 are included in the image information, the object recognition unit 441 recognizes the plurality of objects 10 individually.

  Next, in S103, for the target object 10 recognized by the target object recognition unit 441, the determination points P1 to P8 on the upper surface S1 of the target object 10 are in contact with surrounding objects as illustrated in FIGS. Whether or not the determination is made is determined by the determination unit 442. At this time, if a plurality of objects 10 are recognized by the object recognition unit 441, the determination by the determination unit 442 is performed on each object 10.

  Next, in S104, based on the determination result by the determination unit 442, the setting unit 443 sets each side surface of the object 10 as a finger contactable surface or a finger contact impossible surface. Again, if a plurality of objects 10 are recognized by the object recognition unit 441, the setting by the setting unit 443 is executed for each object 10. Accordingly, in the case of FIGS. 13 to 18, in the target object 10 (10 a, 10 b, 10 c, 10 d), the side surface indicated by the hatched portion is set as the finger contact impossible surface, and the other side surfaces are the finger contact possible surfaces. Will be set as

  Next, in S <b> 105, the gripping surface of the object 10 is determined by the gripping surface determination unit 444 based on the situation of the finger-contactable surface and the finger-contact impossible surface set by the setting unit 443. For example, when the object 10 is sandwiched between the finger parts 21 of the hand mechanism 2, it is preferable that the gripping surface be two gripping surfaces parallel to each other in the object 10. Therefore, for example, in the case of FIG. 14, the side surface S2 and the side surface S4 of the object 10 may be determined as the gripping surface. Further, when the objects 10a, 10b, or 10c in FIGS. 15 and 16 are gripped, the side surface S2 and the side surface S4 may be determined as the gripping surfaces. Also, when the objects 10a and 10c in FIG. 17 or the object 10c in FIG. 18 is gripped, the side surface S2 and the side surface S4 may be determined as the gripping surfaces. Further, when gripping the object 10d in FIG. 17, the side surface S3 and the side surface S5 may be determined as the gripping surface.

  According to this flow, based on image information acquired by imaging an image including the object 10 by the imaging device 8, which side of the object 10 is a finger-contactable plane and which side is finger-contacted. It is possible to appropriately grasp whether it is impossible. In addition, the gripping surface for gripping the object can be suitably determined. Then, the information on the gripping surface of the target object 10 obtained as a result of the image information processing by the image information processing apparatus 44 is used as the target object information together with the information on the shape and dimensions of the target object 10. The information is input to the information acquisition unit 430. Then, the hand control unit 431 controls the hand mechanism 2 based on the object information in order to hold the object 10 by the hand mechanism 2.

In the present embodiment, the determination point setting position on the upper surface S1 of the object 10 is set to the midpoint of each side forming the upper surface S1 and each vertex of the upper surface S1, as shown in FIG. However, the setting position of the determination point on the upper surface S1 of the target object 10 is not necessarily limited to such a position, and may be a predetermined position on each side forming the upper surface S1.

<Example 2>
In the gripping system according to the present embodiment, the configuration of the image information processing apparatus 44 built in the pedestal portion 4 of the robot arm 1 is different from that of the first embodiment. FIG. 20 is a block diagram illustrating each functional unit included in the image information processing apparatus 44 according to the present embodiment. As shown in FIG. 20, in this embodiment, the image information processing apparatus 44 includes a gripping target determining unit 445 instead of the gripping surface determining unit 444 shown in FIG. As with other functional units, the gripping target determining unit 445 is formed by executing a predetermined control program in a computer included in the image information processing apparatus 44.

  Also in the image information processing apparatus 44 according to the present embodiment, with respect to the target object 10 recognized by the target object recognition unit 441, whether or not the determination points P1 to P8 on the upper surface S1 of the target object 10 are in contact with surrounding objects. The determination unit 442 determines whether or not. Furthermore, based on the determination result by the determination unit 442, the setting unit 443 sets each side surface of the object 10 as a finger-contactable surface or a finger-contactless surface. At this time, if a plurality of objects 10 are recognized by the object recognition unit 441, the determination unit 442 determines whether or not each of the determination points P1 to P8 is in contact with a surrounding object, and is set. The setting of the finger contact possible surface and the finger contact impossible surface by the unit 443 is executed for each object 10. In the present embodiment, among the plurality of objects 10 recognized by the object recognition unit 441, which object 10 is the object of the current gripping by the hand mechanism 2 is determined in each object 10. It is determined by the gripping target determination unit 445 based on the situation of the finger contactable surface and the finger contact impossible surface.

  As described above, when the object 10 is to be gripped by the hand mechanism 2, it is necessary to sandwich the object 10 by the finger portion 21 of the hand mechanism 2. In this case, it is preferable that the object 10 is sandwiched between the finger parts 21 after the finger parts 21 are brought into contact with the two side surfaces of the object 10 facing each other. Therefore, in this embodiment, when the object to be grasped by the hand mechanism 2 is determined from the plurality of objects 10, the object 10 having a pair of finger-contactable surfaces that oppose each other is selected as the object to be grasped.

  For example, when the target object 10a, the target object 10b, the target object 10c, and the target object 10d are placed in the state shown in FIG. 17, in the target object 10b and the target object 10c, the side surface S2 and the side surface S4 are mutually connected. It becomes a pair of finger contactable surface which opposes. In the object 10d, the side surface S3 and the side surface S5 are a pair of finger-contactable surfaces that oppose each other. However, the object 10a does not have a pair of finger-contactable surfaces that oppose each other. Therefore, in such a situation, the gripping target determination unit 445 determines the target object 10b, the target object 10c, or the target object 10d as a target to be gripped by the hand mechanism 2. In addition, when the target object 10a, the target object 10c, and the target object 10d are placed in the state shown in FIG. 18, only the target object 10c has a pair of finger-contactable surfaces that oppose each other. (In the object 10c, the side surface S2 and the side surface S4 are a pair of finger-contactable surfaces that oppose each other.) Therefore, in such a situation, the gripping target determination unit 445 determines the target object 10c as a target to be gripped by the hand mechanism 2.

  FIG. 21 is a flowchart illustrating a flow of image information processing performed in the image information processing apparatus 44 according to the present embodiment. This flow of image information processing is realized by executing a predetermined processing program in the image information processing apparatus 44. Note that S101 to S104 in this flow are the same steps as the flow shown in FIG. Therefore, the description about the process in these steps is omitted.

In this flow, the process of S205 is executed after S104. In S205, the setting unit 4
Based on the situation of the finger contactable surface and the finger non-contactable surface of each object 10 set by 43, the object 10 to be grasped this time by the hand mechanism 2 is determined by the grasping object determination unit 445. The

  According to this flow, the object 10 to be gripped by the hand mechanism 2 is suitably determined based on image information acquired by capturing an image including a plurality of objects 10 by the imaging device 8. Can do. Then, the information about the target object 10 to be gripped this time obtained as a result of the image information processing by the image information processing apparatus 44 is used as the target object information together with the information about the shape and dimensions of the target object 10. It is input to 43 object information acquisition unit 430. Then, the hand control unit 431 controls the hand mechanism 2 based on the target object information so as to hold the target object 10 to be gripped this time by the hand mechanism 2.

<Example 3>
In the gripping system according to the present embodiment, the configuration of the image information processing apparatus 44 built in the pedestal portion 4 of the robot arm 1 is different from that of the first embodiment. FIG. 22 is a block diagram illustrating each functional unit included in the image information processing apparatus 44 according to the present embodiment. As shown in FIG. 22, in this embodiment, the image information processing apparatus 44 includes a gripping form determination unit 446 instead of the gripping surface determination unit 444 shown in FIG. As with other functional units, the gripping form determining unit 446 is formed by executing a predetermined control program in a computer included in the image information processing apparatus 44.

  Here, direct gripping and tilting gripping, which are specific examples of gripping forms when the object 10 is gripped by the hand mechanism 2, according to the present embodiment, will be described with reference to FIGS. Here, each finger part 21 of the hand mechanism 2 is referred to as a first finger part 21A, a second finger part 21B, a third finger part 21C, and a fourth finger part 21D, respectively. FIG. 23 is a diagram illustrating the state of the hand mechanism 2 and the target object 10 when the target object 10 is gripped by direct gripping. As shown in FIG. 23, if the placed object 10 is not in contact with any surrounding objects, the object 10 is sandwiched and held by the finger part 21 of the hand mechanism 2 as it is. be able to. In this way, a gripping form in which the object 10 is held by the hand mechanism 2 while being placed is referred to as direct gripping. Such gripping of the object 10 by direct gripping can be performed in the situation where there is at least one pair of finger-contactable surfaces facing each other on the object 10.

  24 to 26 are diagrams showing the state of the hand mechanism 2 and the object 10 when the object 10 is gripped by tilting gripping. 24 and 25 are diagrams illustrating an operation when the object 10 is tilted by the first finger portion 21A of the hand mechanism 2. FIG. FIG. 26 is a diagram illustrating a state where the object 10 is gripped by the second finger part 21B, the third finger part 21C, and the fourth finger part 21D of the hand mechanism 2. As shown in FIGS. 24 to 26, a gripping form in which the object 10 is tilted and then gripped by the hand mechanism 2 is referred to as tilt gripping.

In more detail, in FIG. 24, the target object 10 is mounted side by side in the state which contacted the other target object 10 which adjoins. Even in such a case, the finger part 21 of the hand mechanism 2 can be brought into contact with the upper surface S1 of the object 10. Therefore, first, as shown in FIG. 24, the first finger link portion 211A of the first finger portion 21A is brought into contact with the upper surface S1 of the object 10. At this time, in a state where the first finger portion 21A is in contact with the upper surface S1 of the object 10, the other finger portions 21B, 21C, and 21D in the hand mechanism 2 are not in contact with the object 10. Then, as shown in FIG. 25, the object 10 is tilted forward by the first finger part 21A in a state where the first finger link part 211A is in contact with the upper surface S1 of the object 10. Thus, by changing the posture of the object 10 by the first finger portion 21A, the interval between the object 10 and the adjacent object 10 ′ can be increased. Thereby, in the target object 10, the adjacent target object 1
It is in a state in which the finger part 21 of the hand mechanism 2 can be brought into contact with the side surface that is in contact with 0 ′ and cannot be touched with the finger when it is placed. Then, as shown in FIG. 26, the object is moved by the second finger part 21B, the third finger part 21C, and the fourth finger part 21D in a state where the object 10 is tilted by the first finger part 21A. 10 is sandwiched and held. The gripping of the target object 10 by such tilted gripping is such that at least one finger-contactable surface exists in the target object 10 even if there are no two finger-contactable surfaces facing each other. If it is a situation, it can be carried out by tilting the object 10 toward the finger-contactable surface side.

  As described above, the robot arm 1 according to the present embodiment can grip the object 10 not only by direct gripping but also by tilting gripping. Even when the object 10 cannot be gripped by direct gripping, it may be possible to grip by tilting gripping. Therefore, the gripping mode when the object 10 is gripped by the hand mechanism 2 is determined based on the situation of the finger contactable surface and the finger contact impossible surface of the object 10.

  Here, also in the image information processing apparatus 44 according to the present embodiment, for the target object 10 recognized by the target object recognition unit 441, the determination points P1 to P8 on the upper surface S1 of the target object 10 come into contact with surrounding objects. It is determined by the determination unit 442 whether or not it is present. Furthermore, based on the determination result by the determination unit 442, the setting unit 443 sets each side surface of the object 10 as a finger-contactable surface or a finger-contactless surface. In the present embodiment, the grip form when gripping the object 10 by the hand mechanism 2 is further determined by the grip form determining unit 446 based on the situation of the finger contactable surface and the finger contact impossible surface of the object 10. It is determined.

  For example, when the object 10a, the object 10b, the object 10c, and the object 10d are placed in the state shown in FIG. 17, there is a pair of finger-contactable surfaces that oppose each other. When the object 10b, the object 10c, or the object 10d is the object to be grasped, the grasping form determining unit 446 determines the grasping form to be directly grasped. On the other hand, when the target object 10a, the target object 10b, the target object 10c, and the target object 10d are placed in the state shown in FIG. 17, there is no pair of finger-contactable surfaces that oppose each other. When the object 10a is set as a gripping object, the gripping form determination unit 446 determines the gripping form as tilted gripping. In this case, tilting gripping is performed by tilting the object 10a toward the side surface S2 that is a finger-contactable surface.

  In addition, when the object 10a, the object 10c, and the object 10d are placed in the state shown in FIG. 18, the object 10c having a pair of finger-contactable surfaces that oppose each other is grasped. In the case of a target, the gripping form determination unit 446 determines that the gripping form is direct gripping. On the other hand, when the target object 10a, the target object 10c, and the target object 10d are placed in the state shown in FIG. 18, the target object 10a that does not have a pair of finger-contactable surfaces that oppose each other or When the object 10d is set as a gripping target, the gripping form determination unit 446 determines the gripping form as tilted gripping. At this time, when the target object 10a is set as a gripping target, the target object 10a is tilted and gripped by tilting the target object 10a toward the side surface S2 that is a finger-contactable surface. In addition, when the object 10d is set as a gripping object, the object 10d is tilted and gripped by tilting the object 10d toward the side surface S3 or the side surface S4 that is a finger-contactable surface.

  FIG. 27 is a flowchart illustrating a flow of image information processing performed in the image information processing apparatus 44 according to the present embodiment. This flow of image information processing is realized by executing a predetermined processing program in the image information processing apparatus 44. Note that S101 to S104 in this flow are the same steps as the flow shown in FIG. Therefore, the description about the process in these steps is omitted.

  In this flow, the process of S305 is executed after S104. In S <b> 305, the current gripping form is determined by the gripping form determination unit 446 based on the situation of the finger-contactable surface and the finger-impossible surface on the object 10 set by the setting unit 443. Specifically, if there is a pair of finger-contactable surfaces facing each other on the object 10, the gripping form determining unit 446 determines the gripping form to be directly gripped. In addition, in the object 10, if there is no pair of finger-contactable surfaces facing each other and at least one finger-contactable surface is present, the gripping form determination unit 446 tilts and grips the gripping form. To decide.

  According to this flow, based on image information acquired by capturing an image including the target object 10 by the imaging device 8, a gripping mode for gripping the target object 10 by the hand mechanism 2 is suitably determined. be able to. Then, information regarding the current gripping form obtained as a result of image information processing by the image information processing device 44 is input to the hand control unit 431 of the hand control device 43. And in order to hold | grip the target object 10 by the hand mechanism 2, the hand control part 431 controls the hand mechanism 2 based on the information regarding the input grip form.

  In the first embodiment, the second embodiment, and the third embodiment, the image processing in the image information processing apparatus 44 has been described by taking the case where the shape of the object 10 is a rectangular parallelepiped as an example. However, the shape of the object is not necessarily limited to a rectangular parallelepiped. In other words, if the object is a polyhedron, the surface adjacent to the upper surface of the object can be touched with the finger or the finger contact based on the image information captured from above the object by the method described above. Can be set as impossible surface.

  Further, the imaging direction of the object by the imaging device 8 is not necessarily limited above the object. For example, when the hand mechanism 2 is approached from the side of the object and the object is gripped, the object may be imaged by the imaging device 8 from the side. In this case, the plane that is the target of imaging by the imaging device 8 in the target object is a plane that is located on the side of the target object.

DESCRIPTION OF SYMBOLS 1 ... Robot arm, 2 ... Hand mechanism, 8 ... Imaging device, 20 ... Base part, 21 ... Finger part, 22 ... 1st joint part, 23 ... 2nd Joint part 211 ... 1st finger link part 212 ... 2nd finger link part 213 ... Base end part 3 ... Arm mechanism 30a ... 1st joint part 30b ... -2nd joint part, 30c ... 3rd joint part, 30d ... 4th joint part, 30e ... 5th joint part, 30f ... 6th joint part, 31 ... 1st arm link 32, second arm link portion, 33, third arm link portion, 34, fourth arm link portion, 35, fifth arm link portion, 36, connection member, 4 ... Pedestal part, 42 ... Arm control device, 420 ... Arm control part, 421 ... Motor state quantity acquisition part, 43 ... Han Control device, 430 ... object information acquisition unit, 431 ... hand control unit, 432 ... motor state quantity acquisition unit, 44 ... image information processing device, 440 ... image information acquisition unit, 441 ... object recognition unit, 442 ... determination unit, 443 ... setting unit, 444 ... gripping surface determination unit, 445 ... gripping object determination unit, 446 ... gripping form determination unit, 51 ... 1st motor, 52 ... 2nd motor, 53 ... 3rd motor, 61, 62 ... Bevel gear, 63 ... Worm wheel, 64 ... Worm, 65, 66 ...・ Gear, 70 ... pressure sensor, 100 ... grip system

Claims (10)

A gripping system for gripping an object by a hand mechanism having a plurality of fingers, applied to a gripping system having an imaging device that captures an image including at least one of the objects, and acquired by the imaging device In an image information processing apparatus that processes image information,
The object is a polyhedron, and a predetermined position on each side forming the predetermined plane is preset as a determination point in a predetermined plane to be imaged by the imaging device on the object. ,
A determination unit that determines whether or not the determination point is in contact with a surrounding object for each determination point of the predetermined plane of the object based on image information acquired by the imaging device;
A finger that can be contacted by a finger portion of the hand mechanism on another plane adjacent to the predetermined plane across a side where the determination point is determined to be not in contact with the surrounding object by the determination unit. Other hand planes that are set as contactable surfaces and are adjacent to the predetermined plane across the side where the determination point determined to be in contact with the surrounding object by the determination unit are located on the hand mechanism. An image information processing apparatus comprising: a setting unit configured to set as a finger contact impossible surface that cannot be contacted by the finger unit.   The image information processing apparatus according to claim 1, wherein a midpoint of each side forming the predetermined plane in the object and each vertex of the predetermined plane are set in advance as the determination points.   A gripping surface determination unit that determines a gripping surface of the target object when gripping the target object by the hand mechanism based on the state of the finger-contactable surface and the finger-contactless surface set by the setting unit The image information processing apparatus according to claim 1, further comprising: When a plurality of objects are imaged by the imaging device, the determination unit determines whether each determination point is in contact with the surrounding object, and the setting unit allows the finger contactable surface and the finger contact failure. Possible surface settings are performed for each object,
Of the plurality of objects, an object to be grasped by the hand mechanism is determined based on the state of the finger-contactable surface and the finger-contactless surface set by the setting unit in each object The image information processing apparatus according to claim 1, further comprising a gripping target determining unit that performs the processing.   The apparatus further comprises a gripping form determining unit that determines a gripping form when the object is gripped by the hand mechanism based on the state of the finger contactable surface and the finger noncontactable surface set by the setting unit. The image information processing apparatus according to claim 1. The hand mechanism;
The imaging device;
An image information processing apparatus according to any one of claims 3 to 5,
And a hand control device that controls the hand mechanism based on a result of image information processing by the image information processing device. In a gripping system for gripping an object by a hand mechanism having a plurality of fingers, an image information processing method for processing image information acquired by an imaging device that captures an image including at least one object,
The object is a polyhedron, and a predetermined position on each side forming the predetermined plane is preset as a determination point in a predetermined plane to be imaged by the imaging device in the object,
A determination step of determining whether or not the determination point is in contact with a surrounding object for each determination point of the predetermined plane of the object based on image information acquired by the imaging device;
A finger that can be contacted by a finger portion of the hand mechanism on another plane adjacent to the predetermined plane across a side where the determination point is determined to be not in contact with the surrounding object by the determination unit. Other hand planes that are set as contactable surfaces and are adjacent to the predetermined plane across the side where the determination point determined to be in contact with the surrounding object by the determination unit are located on the hand mechanism. A setting step of setting as a finger non-contactable surface that cannot be touched by the finger.   A gripping surface determination step for determining a gripping surface of the target object when the target object is gripped by the hand mechanism based on the state of the finger contactable surface and the finger noncontactable surface set in the setting step. The image information processing method according to claim 7, further comprising: When a plurality of objects are imaged by the imaging device, in the determination step, it is determined whether or not each determination point is in contact with the surrounding object for each object, and in the setting step The setting of the finger contactable surface and the finger contact impossible surface is performed for each object,
Of the plurality of objects, an object to be grasped by the hand mechanism is determined based on the state of the finger contactable surface and the finger non-contactable surface set in the setting step in each object. The image information processing method according to claim 7, further comprising a gripping target determination step. A gripping mode determining step of determining a gripping mode when the object is gripped by the hand mechanism based on the state of the finger contactable surface and the finger non-contactable surface set in the setting step. Item 8. The image information processing method according to Item 7.

Images