JPH05168332A - Fruit harvesting robot - Google Patents

Abstract

(57) [Summary] [Objective] In a fruit harvesting robot, it is easy to align the center of the camera image of the visual device with the scanning center of the distance sensor so that good fruit picking work can be performed. A fruit harvesting robot comprising a manipulator capable of moving a fruit picking hand in three-dimensional directions and a visual device for detecting fruits, the visual device comprising a camera for imaging an object and a predetermined range. And a distance sensor that measures the distance to the object by scanning vertically and horizontally, the image of the camera is configured to identify the crop based on the processed image obtained from the measurement result of the distance sensor, The camera and the distance sensor are arranged so that the center of the image of the camera and the center of the scanning range of the distance sensor can be matched only by activating a predetermined one movable part of the manipulator, and the center of the processed image is The reference point of the hand part in the initial posture of the manipulator is made to match.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a fruit harvesting robot for automatically harvesting harvestable fruits.

[0002]

[Prior Art] While self-propelled along the planting line of cultivated crops,
A fruit harvesting robot has been developed which finds out a mature fruit that can be harvested with a visual device and automatically harvests it by a manipulator having a hand for picking fruits.
An image sensor camera (CCD camera) having a filter that passes only the light of the hue component peculiar to the fruit as a visual device for searching the harvested product provided in this kind of fruit harvesting robot
And a distance sensor (PSD) that scans a certain range vertically and horizontally to measure the distance to the object.
There is a configuration in which the U analyzes the image by the CCD based on the measurement result of the PSD, and discriminates between the fruit and a foreign substance such as a leaf overlapping with the fruit.

[0003]

Conventionally, in the visual device having the above-described structure, the focus of the CCD (center of imaging) and the focus of PSD (center of the scanning range) are deviated by the amount of deviation between the mounting position of the CCD and the mounting position of the PSD. Therefore, the image processing is performed after the correction is made so that the detection ranges of the both agree with each other. Further, the obtained processed image is processed so as to coincide with the reference point of the hand unit, and is used as position data when operating the manipulator. However, such a correction process is complicated, and there is a problem that the operation position of the hand portion is slightly deviated from the actual fruit position due to an error in the correction process.

[0004]

In order to solve the above problems, the present invention has the following constitution. That is, the visual device of the fruit harvesting robot according to the present invention is a fruit harvesting robot having a manipulator capable of moving the fruit picking hand part in a three-dimensional direction, and a visual device for detecting fruit, wherein the visual device is A camera that captures an object and a distance sensor that vertically and horizontally scans a certain range to measure the distance to the object, and the image of the camera is a processed image obtained from the measurement result of the distance sensor. The camera and the distance sensor are arranged so that the center of the image of the camera and the center of the scanning range of the distance sensor can be matched only by operating one movable part in the manipulator. It is arranged such that the center of the processed image coincides with the reference point of the hand portion in the manipulator initial posture. To have.

[0005]

According to the present invention, when the image of the camera is processed based on the detection result of the distance sensor, the position of the center of the image of the camera and the center of the scanning range of the distance sensor is adjusted by one predetermined movement in the manipulator. All you have to do is activate the unit. Therefore, it becomes easy to move the manipulator so that the image pickup center of the camera actually coincides with the center of the scanning range of the distance sensor to perform image processing.
There is no need to perform correction processing. Further, since the center of the obtained processed image coincides with the reference point of the hand portion at the manipulator initial position, it can be used as it is for the operation of the manipulator without performing correction processing on the data of the processed image. In this way, when aligning the center of the image of the camera with the center of the scanning range of the distance sensor and aligning the center of the processed image with the center of the hand part,
Since it is not necessary to perform correction processing on the input data, the processing is simple and the error in fruit position detection can be reduced.

[0006]

EXAMPLE FIG. 1 and FIG. 2 show the usage state of a fruit harvesting robot for cucumbers. The fruit harvesting robot 1 includes an electric traveling unit 2 as a moving means, and a manipulator 3 for fruit harvesting, a visual device 5 for searching for a harvested product, a fruit storage unit 6 and the like are installed on the traveling unit 2. Become. In a cultivating field where the fruit harvesting robot 1 is used, a cucumber tree 8 is supported by a support 7 provided obliquely.

As shown in FIGS. 3 and 4, the manipulator 3 has a tilted frame 10 tilted so as to be substantially parallel to the tilt of the tree 8 facing it, and a guide rail 11 of the tilted frame.
A base 12 mounted to be movable up and down along the base 12, a main body 13 rotatably provided on the base in a horizontal plane, an articulated arm 14 provided on the main body, and an articulated arm of the articulated arm. It comprises a fruit picking hand portion 15 provided at the tip. The tilt frame 10 is
The hinge 16 supports the traveling unit 2 and the back side of the traveling unit 2 is supported by a support link 17. The lower end of the support link 17 is fixed to a proper position of the elongated hole 18, and the inclination angle of the inclined frame 10 can be arbitrarily adjusted by changing the fixing position. It is preferable that the inclination of the inclined frame 10 be the same as that of the support 7.

The articulated arm 14 is composed of the upper arm 2 on the main body side.
0 and a forearm 21 on the hand side, the upper arm 20 pivots up and down around the shoulder 22 as in the case of a human arm, and the wrist part 2 bends and stretches as a whole, and the wrist 2
The hand portion 15 is adapted to rotate up and down around the center 4. When the articulated arm 14 is bent, the elbow portion 23 bends so as to come to the lower side. The upper arm portion 20 and the forearm portion 21 are provided with two plate members 26, 2 at a predetermined interval.
6, 27, 27 are formed to face each other, and when the articulated arm 14 is bent to the maximum extent, the upper arm portion 20 and the forearm portion 21 overlap with each other as shown by a two-dot chain line in FIG.
The wrist portion 24 is located inside the shoulder portion 22. This state is the initial posture of the manipulator.

As described above, since the wrist portion 24 is located inside the shoulder portion 22 in the initial posture, fruits located near the fruit harvesting robot 1 can also be harvested. Also, the articulated arm 14 and the hand portion 15 do not come into contact with the cultivated plant during traveling,
It is possible to prevent damage to cultivated plants. Further, since the moment of inertia becomes small, there is also an advantage that the later-described motor 26 for turning the main body 13 can be made compact.

The manipulator 3 is constructed as described above, and the height of the hand portion 15 in the vertical direction (Z-axis direction) is adjusted by moving the base 12 up and down along the guide rails 11 to make the base 12 Main body 13 centered on the Z-axis above
The horizontal angle (Y-axis direction) of the hand unit 15 is adjusted by rotating the hand in the horizontal plane.
By extending and contracting 4 the front and rear direction (X
Adjust the distance in the axial direction). That is, the manipulator 3 can adjust the position of the hand unit 15 in the three-dimensional direction. In the figure, reference numeral 25 is a base lifting motor, 26 is a main body turning motor, 27 is a shoulder motor, 28 is an elbow motor, and 29 is a wrist motor.

As shown in FIGS. 5 to 13, the hand portion 15 includes a pair of left and right grasping portions 31, (L, R), a fruit pattern position detecting portion 32 (L, R) and a cutter 33 (L, R). ) Has. The entire hand portion 15 can be rolled around the rolling shaft 35. Reference numeral 36 in the figure
Is the rolling drive motor.

The grasping portion 31 (L, R) is formed into a shape suitable for grasping the cucumber by sandwiching it from both sides, and rubber cushion materials 44, 44 are attached to the grasping surface. The left and right grasping portions 31 (L, R) are slidably supported in the left and right directions along the guide rods 40 (L, R), and the pinion 42 that meshes with the rack 41 (L, R) attached to the base portion. Are driven by a motor 43 so as to approach or separate from each other.

The fruit pattern position detectors 32 (L, R) are plate-like members which are arranged so as to face each other.
It is slidably supported left and right along (L, R),
The springs 47 (L, R) urge the springs 47 (L, R) toward each other (inward). Fruit position detecting unit 32
The opening operation of (L, R) is performed by the grasping portion 31 (L, R
The closing operation is performed by the action of the spring 47 (L, R). With this configuration, the number of actuators in the hand unit 15 can be reduced by one, and the size and weight of the hand unit 15 can be reduced. The left and right fruit pattern position detection units 32 (L, R) are applied to the outer surface of the fruit or the fruit pattern, and the fruit pattern is detected from the opening degree at that time. The degree of opening is measured by one of the peduncle position detection units 32 (L)
The slide amount is detected by the potentiometer 48.

The cutter portions 33 (L, R) are formed such that one (R) has a blade body whose inner end portion is sharply formed, and the other (L) has a support body for receiving a peduncle from the opposite side. Both cutter parts 33 (L, R) are respectively guide rods 50 (L, R).
It is slidably supported right and left along R), and its base is screwed to the screw portion 52 (L, R) of the screw shaft 51. The screw parts 52 (L, R) are threaded in opposite directions, and the left and right cutters 33 (L, R) are driven in the left and right directions by rotating the screw shaft 51 with a motor 53.

Further, the fruit pattern position detecting portion 32 (L, R) and the cutter portion 33 (L, R) are attached to a common mounting frame 55, and the mounting frame 55 can be moved up and down. The range of vertical movement is about 30 mm at maximum. The structure of the vertical movement mechanism is such that the mounting frame 55 is slidably supported by the guide rod 56 in the vertical direction, and the screw hole 57 formed in the mounting frame 55.
The screw shaft 58 is screwed together, and the screw shaft 58 is rotated by the motor 59.

As shown in the block diagram of FIG. 14, the visual device 5 includes an image sensor camera (CCD) 60 for picking up only a hue component of green and a vertical range and a horizontal range within a predetermined range to reach an object. Distance sensor (P
SD) 61 and a strobe 62 for irradiating an object with light,
And a CPU 63 that analyzes and processes input signals from the CCD 60 and PSD 61. The CCD 60 is provided on the upper portion of the manipulator body 13, and the center of the lens is aligned with the reference point (center) of the hand unit 15 in the initial state. The PSD 61 is provided at the same height as the CCD 60, and is oriented laterally so that the center of the scanning range is at a position of 90 degrees with respect to the CCD 60 about the Z axis. Further, the CPU 63 is built in the main body 13.

The distance sensor (PSD) 61 has the structure shown in FIG. The light emitted from the light projecting device 64 is reflected by the reflecting mirror 65 to illuminate the object 66, and the reflected light is reflected by the reflecting mirror 67 to be received by the light receiving device 68. Since the distance between the reflecting mirrors 41 and 43 is constant, the distance to the object 66 is measured based on the principle of triangulation. Horizontal scanning is performed by changing the angles of the reflecting mirrors 65 and 68 to the stepping motors M ₁ and M _2, and vertical scanning is performed by rotating the entire case 69 by the stepping motor M ₃ .

The search for the crops by the visual device 5 is performed in the order shown in the flowchart of FIG. That is, the main body 13 is made to face the target plant, the object is imaged by the CCD 60, the main body 13 is rotated 90 degrees from that state, and the distance to the target is measured by the PSD 61. If the CCD input image is as shown in FIG. 17 and the luminance distribution of the pixels on the line aa is as shown in FIG. 18, the pixels whose luminance is at or above the binarization level (the hatched portion in FIG. 18) are displayed. Pixels below the binarization level of "1" are processed as "0" to obtain the processing screen shown in FIG. Then, the vertical length / horizontal width ratio of the object on this processing screen is obtained, and if the ratio is within the specified range, the object is determined to be cucumber. The standard vertical length is 20 cm, and the horizontal width is 2.2 cm.
Is. Next, the center coordinates of the cucumber in the image are obtained based on the measurement result of the PSD 61, and the manipulator 3 is operated in accordance with the center coordinate.

Since the image of the CCD 60 and the measurement result of the PSD 61 are for the same object, the CPU 63 can analyze the data as it is without performing a correction process. Further, since the center of the processed image thus obtained coincides with the center of the hand unit 15 in the initial state, the position data of the processed image can be used as it is as the target position data of the manipulator 3.

Next, the operation of the fruit harvesting robot 1 will be described with reference to the flowchart of FIG. When starting the work, hold the manipulator 3 in the initial position,
Along the line of the cultivated plant, one of the CCDs 60 in the traveling unit 2
The crops are searched according to the search procedure of the visual device 5 while moving by the screen width. After finding the harvestable fruits, the shoulder section 22, the elbow section 23, and the wrist section 2
4 is operated appropriately to bring the hand portion 15 close to the target fruit. Then, the hand portion 15 is operated in the following order to pick the target fruit.

In the initial state, the grasping parts 31, 31 and the fruit pattern position detecting parts 32, 32 are both closed. First, the grasping units 31 and 31 are opened. Engaging piece 31 of grasping portion 31, 31
a and 31a are engaging pieces 32 of the fruit pattern position detecting portions 32 and 32.
Since they are engaged with the insides of a and 32a, the engagement piece / fat pattern position detecting portions 32 and 32 are also opened. When the fruits are located inside the grasping portions 31, 31, the grasping portions 31, 31 close to grasp the fruits. Since the cushioning material 44, 44 is attached to the grasping surface, the grasping part and the cucumber elastically contact each other, and the fruit is less damaged, and the contact area between the two is wide, which is convenient for grasping the cucumber. Is.
Subsequently, the fruit pattern position detection units 32, 32 and the cutters 33, 33
Moves up. Engaging pieces 31a, 31a and engaging pieces 32a, 3
Since the engagement of 2a is disengaged, the fruit pattern position detecting portions 32, 32 are pressed against the outer peripheral surface of the fruit by the force of the springs 47, 47, and the fruit pattern position detecting portions 32, 32 are moved by the hand portion 15 as they rise. Move along the perimeter. When the fruit pattern position detectors 32, 32 detect a fruit pattern, the cutters 33, 3
3 operates to cut the peduncle. Both fruit pattern position detector 32
Since (L, R) is biased by springs 47 (L, R) in the approaching direction, it reliably contacts the outer surface of the fruit or the fruit pattern, and an accurate detection result can be obtained.

As shown in FIG. 21, the cucumber fruit is divided into three parts, a fruit base portion A, a central portion B, and a fruit top portion C, and the relation between the surface compressive force and the surface displacement is experimentally investigated. In the central portion B and the fruit apex C, permanent compression occurs when the compressive force per cm ² becomes 3 to 4 N, but the fruit base A
It was found that the compression force per cm ² can withstand up to about 10 N (see FIG. 22). This is because the fruit base A has no seeds and is thus hard. Therefore, the grasping unit 31,
It is preferable to grasp the fruit base portion A at 31 and set the driving force of the motor so that the grasping force is about 6N. When the gripping force is 6 N, it is possible to grip about 1 kg of fruit due to the relationship between the contact area and the friction coefficient. Since the average weight of cucumbers is about 200 g, most cucumbers can be grasped with certainty.

A corrugated cardboard box 70 (L, M, S) for each size is attached to the fruit storage unit 6. In the illustrated example, the upper row is for L size, the interruption is for M size, and the lower row is for S size. When the crops are identified by the visual device 5, the shape, length, and thickness of the cucumbers are discriminated, and the cucumbers are classified into the above-mentioned three sizes. Then, the harvested cucumbers are sorted and stored in each cardboard box according to the selected size. That is, this fruit harvesting robot automatically performs the processes from harvesting to packaging in a single unit during the process from harvesting to shipping shown in FIG. This allows
Eliminates the need for manual selection and packaging, greatly improving work efficiency.

As shown in FIG. 2, a harvesting-carrying robot 71 capable of loading a plurality of shipping cardboard boxes is separately provided, and the harvesting-carrying robot 71 is made to follow the fruit harvesting robot 1. If so, more cucumbers can be harvested in one harvesting operation, which is more efficient.

A support net shown in FIG. 24 is used as a support for cultivated plants. The support net 73 is formed by vertically arranging thin rods 74, ... Made of metal or resin at equal intervals (about 7 cm), and connecting the upper and lower ends thereof with strings 75, 75. When a support without such a horizontal rail is used, the fruit does not get caught on the horizontal rail, so that the fruit naturally hangs due to gravity, and a good growth condition is obtained. Also, since the vertical bars are arranged at equal intervals, the main stem 7
With the support of 6, the side buds 77, ... Can be attracted, which simplifies management.

As shown in FIGS. 25 and 26,
The CCD 60 and the PSD 61 are vertically arranged side by side at the same position in a plan view, and the substrate 12 is moved up and down so that the image center of the CCD 60 and the PS
You may make it match the scanning center of D61.

[0027]

As described above, according to the fruit harvesting robot of the present invention, the center of the image pickup of the camera and the center of the scanning range of the distance sensor are aligned, and the center of the processed image is aligned with the reference point of the hand portion. At this time, since it is not necessary to perform the correction process on the input data, the process is simple and the error in the fruit position detection can be reduced.

[Brief description of drawings]

FIG. 1 is a front view showing a usage state of a fruit harvesting robot.

FIG. 2 is a side view of a fruit harvesting robot.

FIG. 3 is a side view of the manipulator.

FIG. 4 is a plan view of the manipulator.

FIG. 5 is an external side view of a hand portion.

FIG. 6 is an external plan view of a hand unit.

FIG. 7 is a side view of a hand portion mainly showing a grasping portion.

FIG. 8 is a plan view of a hand portion mainly showing a grasping portion.

FIG. 9 is a side view of a hand portion mainly showing a fruit pattern position detection unit.

FIG. 10 is a plan view of a hand part mainly showing a fruit pattern position detection part.

FIG. 11 is a side view of a hand portion mainly showing a cutter portion.

FIG. 12 is a plan view of a hand portion mainly showing a cutter portion.

FIG. 13 is a side view of a hand part mainly showing a vertical movement mechanism of a fruit pattern position detection part and a cutter part.

FIG. 14 is a block diagram of a visual device.

FIG. 15 is an explanatory diagram of a distance sensor.

FIG. 16 is a flowchart showing the control of the visual device.

FIG. 17 is a diagram showing a camera input image.

FIG. 18 is a diagram showing a luminance distribution.

FIG. 19 is a diagram showing a processed image.

FIG. 20 is a flowchart showing the operation of the fruit harvesting robot.

FIG. 21 is an explanatory diagram showing cucumber parts.

FIG. 22 is a diagram showing the results of an experiment for measuring the physical properties of cucumber fruits.

FIG. 23 is a flowchart of processes from harvesting to shipping.

FIG. 24 is a view of a support net.

FIG. 25 is a front view of a different fruit harvesting robot.

26 is a plan view of the fruit harvesting robot shown in FIG. 25. FIG.

[Explanation of symbols]

 1 Fruit harvesting robot 3 Manipulator 5 Visual device 15 Hand part 60 Image sensor camera 61 Distance sensor

Claims (1)

[Claims] 1. A fruit harvesting robot comprising a manipulator capable of moving a fruit picking hand part in a three-dimensional direction, and a visual device for detecting fruits, wherein the visual device is a camera for imaging an object and a fixed amount. A distance sensor that vertically and horizontally scans a range to measure a distance to an object, and an image of the camera is identified based on a processed image obtained from a measurement result of the distance sensor. In addition, the camera and the distance sensor are arranged so that the center of the image of the camera and the center of the scanning range of the distance sensor can be matched only by operating one movable part of the manipulator, and the center of the processed image. The fruit harvesting robot is characterized in that it is made to coincide with the reference point of the hand part in the initial posture of the manipulator.