US20220183230A1

US20220183230A1 - Harvester with robotic gripping capabilities

Info

Publication number: US20220183230A1
Application number: US17/425,536
Authority: US
Inventors: Frank FAULRING; Carlos Ramirez; Jason FAULRING; Mark Keyser; Jacob GOELLNER
Original assignee: Ceres Innovation LLC
Current assignee: Ceres Innovation LLC
Priority date: 2019-01-24
Filing date: 2020-01-23
Publication date: 2022-06-16
Also published as: WO2020154515A1; US20220087106A1; WO2020154473A1; WO2020154646A1; US20240306544A1; US20220078972A1; US12029166B2

Abstract

Systems and methods here may include a vehicle with automated subcomponents for harvesting delicate targets such as agriculture. In some examples, the vehicle includes a targeting subcomponent and a harvesting subcomponent. In some examples, the harvesting subcomponent includes vacuum features which gently attach to target agriculture to secure it. In some examples, the harvesting subcomponent includes padded spoons to grasp and remove the target agriculture from the foliage.

Description

CROSS REFERENCE

This application relates to and claims priority to U.S. Provisional application 62/796,319 filed Jan. 24, 2019 the entirety of which is hereby incorporated by reference.

TECHNICAL FIELD

This application relates to the field of automated agricultural harvesting equipment using robotic gripping assemblies, mobile harvesting units, produce handling, and various combinations of related technologies.

BACKGROUND

The agriculture industry is highly reliant on human pickers to harvest a number of produce, including berries such as strawberries. The reason human pickers are still used today, despite the technological advancements available, is because of the difficulty of identifying a target such as a berry in a field, that is ready to be picked, reaching through the foliage of the plant to grasp that berry, and then carefully removing that berry without damaging it, to package and sell immediately. The harsh conditions of the field make this a difficult task to automate and roboticize.
Current automatic harvesting of such delicate and difficult to grasp agricultural targets such as berries, while operating in a harsh outdoor environment did not exist before this application, let alone systems the capabilities described herein.

SUMMARY

Systems and methods here may include a vehicle having various subcomponents for harvesting delicate agricultural items such as berries. In some examples, the subcomponents may be automated. In some examples, the vehicle may include a targeting subcomponent and a harvesting subcomponent. In some examples, the targeting subcomponent utilizes multiple cameras to create three dimensional maps of the target and target areas sometimes including the agricultural foliage. In some examples, the targeting subcomponent may include any of various cameras, sensors, or other targeting features to locate and map targets in an automated or semi-automated manner. The system may then determine coordinates of the mapped targets to be passed to the harvesting subcomponent. In some examples, the harvesting subcomponent may include vacuum features which help a nozzle attach to an agriculture target for harvesting. In some examples, the harvesting subcomponent includes padded spoons to aid in removal of the targeted agriculture from the plant, including in some examples, a stem.
Systems and methods for harvesting agriculture describe here may include a traversing machine with a frame, an articulating robotic arm attached to the frame, and a computer system attached on the frame with a processor and memory in communication with the articulating robotic arm, the articulating robotic arm including at least two joints and at least one picker subassembly, the picker subassembly including at least one actuator in communication with the computer system, in some examples, the picker subassembly includes a vacuum subassembly, the vacuum subassembly in communication with the computer system, the vacuum subassembly coupled to a nozzle with a terminal end, wherein the terminal nozzle end includes a flexible baffle section, in some examples, the picker subassembly further includes two grappler spoons, the grappler spoons configured to pinch together toward the vacuum nozzle to secure a target by the actuator, in response to commands from the computer system.
In some examples, additionally or alternatively, the picker subassembly vacuum nozzle is mounted on an extender actuator, in communication with the computer system, the extender actuator configured to extend away from the picker subassembly and back toward the picker subassembly, the grappler spoons configured to pinch toward the nozzle terminal end when the vacuum nozzle is retracted. In some examples, additionally or alternatively, the flexible baffle section is removable and friction fit to the picker subassembly and made of silicone. In some examples, additionally or alternatively, the grappler spoons are each attached to the picker subassembly by a flange and include at least one rim, and in some examples, the grappler spoons each include a resilient membrane stretched over the at least one rim. In some examples, additionally or alternatively, the grappler spoon membrane is removable and friction fit to the rim and made of silicone. In some examples, additionally or alternatively, the vacuum subassembly nozzle terminal end is generally round in cross section and includes a resilient end membrane with apertures to allow air to flow. In some examples, additionally or alternatively, the vacuum assembly nozzle terminal end apertures include at least four radially extending slots configured in a middle of the round resilient membrane. In some examples, additionally or alternatively, the picker subassembly is configured to twist in relation to the robotic arm, such that the twist might break a target stem when grappled by the terminal nozzle end and spoons. In some examples, additionally or alternatively, the picker subassembly includes a stem cutting saw configured around the picker subassembly, the stem cutting saw configured to slide around the picker subassembly and spin, relative to the picker subassembly. In some examples, additionally or alternatively, at least two stereo cameras may be mounted to the frame, in communication with the computer system, configured to send image data regarding potential agricultural targets to the computer system for analysis.
Systems and methods for harvesting agriculture, as described here, may include a traversing machine with a frame, a computer system with a processor and memory mounted to the frame, an articulating robotic arm in communication with the computer system, the articulating robotic arm mounted to the frame at a first end, a pair of pincher spoons mounted to a second end of the articulating robotic arm, the pair of pincher spoons in communication with a pinching actuator, the pinching actuator in communication with the computer system and configured to pinch the pair of pincher spoons toward one another, a vacuum system including a pump and a hose, the pump in communication with the computer system; the hose including a terminal end mounted at the second end of the articulating robotic arm, between the pair of pincher spoons. In some examples, additionally or alternatively, the terminal end of the vacuum hose includes a pliable baffle configured to flex in use. In some examples, additionally or alternatively, the pliable baffle is removably attached at a first end to the terminal end of the vacuum hose. In some examples, additionally or alternatively, the terminal end of the vacuum hose includes an actuator and extendible section configured to extend and retract between the pair of pincher spoons, the actuator in communication with the computer system. In some examples, additionally or alternatively, the pliable baffle includes a second end with slots configured to allow air to pass. In some examples, additionally or alternatively, the slots in the pliable baffle second end are radially configured or concentrically configured. In some examples, additionally or alternatively, the pincher spoons each include an inner and outer rim of pliable material. In some examples, additionally or alternatively, the pair of pincher spoons include a removable pliable membrane. In some examples, additionally or alternatively, the limit of retraction of the terminal end of the vacuum hose is past and above the pair of pincher spoons.
Methods and systems of harvesting agriculture as described here may include traversing a harvester machine down a planter bed row of agriculture, receiving, at a computer system on the harvester machine, image data of the agriculture, determining, at the computer system, likely targets for harvesting from the image data, sending coordinates of the likely targets to a robotic arm attached at a first end to the harvester machine, sending harvest commands to a picker assembly mounted at a second end of the robotic arm, the harvest commands including activating a vacuum pump to draw air through a vacuum tube with a vacuum terminal end on the picker assembly, and actuating pinching of two pincher spoons mounted around the vacuum terminal end. In some examples, additionally or alternatively, the vacuum terminal end includes a flexible baffle section with apertures to allow air to pass through the vacuum tube.

BRIEF DESCRIPTION OF THE DRAWINGS

For a fuller understanding of the nature and objects of the disclosure, reference should be made to the following detailed description, taken in connection with the accompanying drawings, in which:

FIG. 1 is a diagram showing example mobile vehicle examples as described in the embodiments disclosed herein.

FIG. 2 are diagrams showing example robotic arm examples as described in the embodiments disclosed herein.

FIG. 3 is a diagram showing example picker head example details as described in the embodiments disclosed herein.

FIGS. 4A, 4B and 4C are diagrams showing example extraction hardware example steps as described in the embodiments disclosed herein.

FIG. 5 is a diagram showing an example bellows interacting with a target as described in the embodiments disclosed herein.

FIGS. 6A, 6B and 6C are diagrams showing example cutaways of bellows construction as described in the embodiments disclosed herein.

FIGS. 7A, 7B, 7C, and 7D are diagrams showing example bellows interacting with differently sized targets as described in the embodiments disclosed herein.

FIGS. 8A, 8B and 8C are diagrams showing example bellows construction as described in the embodiments disclosed herein.

FIG. 9 is a diagram showing example spoon grappler construction as described in the embodiments disclosed herein.

FIG. 10 is a diagram showing more example spoon grappler construction as described in the embodiments disclosed herein.

FIG. 11 is a diagram showing example spoon grappler cutaway construction as described in the embodiments disclosed herein.

FIG. 12 is a diagram showing an example spoon grappler interacting with a target as described in the embodiments disclosed herein.

FIG. 13 is a diagram showing more example spoon grappler construction as described in the embodiments disclosed herein.

FIG. 14 is a diagram showing an example picker head with a stem trimmer as described in the embodiments disclosed herein.

FIG. 15 is an example computer system which may be used in the embodiments disclosed herein.

DETAILED DESCRIPTION

Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous specific details are set forth in order to provide a sufficient understanding of the subject matter presented herein. But it will be apparent to one of ordinary skill in the art that the subject matter may be practiced without these specific details. Moreover, the particular embodiments described herein are provided by way of example and should not be used to limit the scope of the invention to these particular embodiments. In other instances, well-known data structures, timing protocols, software operations, procedures, and components have not been described in detail so as not to unnecessarily obscure aspects of the embodiments of the invention.

Overview

The systems and methods described here include an automated/semi-automated system with machine(s) that is/are capable of harvesting agricultural targets such as berries without human hands touching the plants or targets themselves. Example overall systems may include subcomponents such as a seeker or sensor subsystem to find and locate the targets, that works with and informs a picking subsystem to harvest the targets. The overall system(s) may be mounted on wheels and/or tracks to advance down a row of targets such as agricultural produce so that the seeker subassembly may identify and map the targets while the picker subsystem is used to harvest, gather, and move the targets.
In some example embodiments, additionally or alternatively, the seeker subassembly includes a camera or multi-camera system used by a remote operator to locate target berries and three dimensionally map them. In such examples, these mapped coordinates may then be queued for harvesting. Additionally or alternatively, in some examples, the harvester subassembly is then able to follow the seeker subassembly and harvest the berries whose mapped locations are queued by the seeker subassembly. In some example embodiments, the harvester subassembly includes at least one robotic arm with multiple degrees of freedom capable of reaching into the foliage of a plant and extracting a target such as a berry. In some examples, the extraction is through a vacuum system. In some examples, additionally or alternatively, the extractions is augmented by a padded spoon grasper, capable of twisting and snapping a berry stem.
It should be noted that the examples used here describing berry harvesting, or even strawberry harvesting in the written description and/or figures is not intended to be limiting and is merely used as an example. The agricultural targets to which the systems here may identify, map, and ultimately harvest may be any sort including but not limited to berries such as strawberries, blackberries, blueberries, and raspberries, other examples include grapes, figs, kiwi, dragon fruit, or other fruits. Vegetables may be harvested as well, such as Brussel sprouts, tomatoes, peppers, beans, peas, broccoli, cauliflower, or other vegetable. Any type of agricultural target may be harvested using the systems described herein. Additionally or alternatively, the systems and methods here may be used to target and gather non-agricultural items such as garbage, or be used to take scientific samples such as rocks or minerals in environments or situations where it may be advantageous to avoid human contact or interaction.
Harvester Subassembly
In some example embodiments, a harvesting subassembly is included as its own separate vehicle system from the seeker subassembly. Additionally or alternatively, in some examples, the harvesting subassembly may be in communication with or connected to the seeker subassembly. In some examples, seeker subcomponents are integrated into the harvesting assembly and one machine incorporates all of the features described herein. The harvesting subassembly may include any number of features that allow for autonomous, semi-autonomous, or human operable harvesting of delicate target agriculture such as berries, as described herein.
In some examples, either harvesting or seeker subassembly may be mounted on its own vehicle subassembly with wheels and/or tracks or combination of both, to traverse down a row of agriculture with the seeker subassembly identifying and mapping the target berries and the harvester subassembly gathering targets.
FIG. 1 shows examples of the overall traversing machine to which any of the various subassemblies may be attached and/or coupled. In the example, the main traversing subassembly 152 includes various portions mounted to it including main driving and/or steering wheels 154 and in some examples, guide wheels 156. In some examples, guide wheels 156 may be canted outward in order to support traversing a mound 101 should a planting bed mound be configured for the plants. In some examples, tank treads or tracks may be used instead of wheels, and/or a combination of wheels and tracks may be used, the drawings depicting wheels are not intended to be limiting.
In some examples, any number of robotic arms 160 may be mounted to any of various frame portions 153 and/or chassis portions 155 that comprise the overall traversing subassembly 152. It should be noted that many variations of robotic arms 160 may be used in the systems described here, including but not limited to robotic wrists with link and joint combinations with linear and rotational links, gantry robots with linear joints, cylindrical robots connected to rotary base joints, polar robots for twisting, and/or jointed-arm or articulating robots with twisting joints and rotary joints. Any combination of these or other robotic assemblies 160 may be used on the systems described herein to manipulate a picker head and/or sensors for harvesting agricultural targets as described.
In some examples, the robotic arm(s) 160 may include at least one picker head, at least one sensor, at least one light system, and/or a combination of picker heads, sensors, and/or lights. In some examples, the sensors are mounted to the frame 153 of the harvester.
For example purposes, the range 195 of the robotic arm 160 is shown in the FIG. 1 to show that the robotic arm 160 may reach different sides of a particular row mound 101 where targets may be found and an accumulator for processing targets, such as the example traversing conveyor 178. It should be noted that the system in FIG. 1, is shown straddling one plant bed row. In some examples, one system may straddle two, three, or more plant bed rows. In some examples, wider or narrower plant bed rows may be straddled and the example in FIG. 1, is merely intended to be an example, and not limiting. By making the system wider to straddle a second row, two sets of arms 160 may be used to pick two rows, or three, or four, or more number of arms, commensurate to the task of picking.
In some examples, multiple robotic arms 160 may be fit onto one overall traversing vehicle 152. For example, systems may include a primary picker assembly with a clean-up/redundant picker assembly which operates behind the primary setup. In those examples, up to eight picker arms may be employed, four on the primary and four on the clean-up assembly, with one or two arms operating on each side of two rows. The clean-up system may operate in the same way that the primary system operates, to find targets that the primary system did not harvest, and/or to operate as a redundancy should one or more arms on the primary system malfunction.
In some example embodiments, the harvesting subassembly may include at least one robotic arm 160 with joints that allow for multiple degrees of freedom. Such arms 160 may be configured to maneuver around and in foliage of a target plant to extract the target agriculture such as but not limited to berries of any sort.
In examples where targets are fruit plants which are harvested many multiple times during a single growing season, often multiple times per week, leaving fruit on a fruit plant may curtail the productivity of the plant. If the plant senses that it still has fruit on it, even rotting or deformed fruit, it may not produce more fruit. This would limit production, so one of the goals of the systems and methods here is to remove all of the fruit when ripe, or when it should be removed to make the plant replace rotting or deformed fruit. As the bed rows may allow for some fruit to drape over the side of the plastic, and become easily exposed to viewing, other fruit may grow under the foliage, or on top of the bed row tops and be obscured by foliage. Therefore, to find and harvest as much fruit from each plant as possible, it may be necessary to maneuver the foliage to better view and/or harvest fruit targets.
In some examples, foliage moving arms may be included to alter, move, displace, and or otherwise gently maneuver the foliage of the plant to better expose the targets such as fruit berries to be picked. In such examples, a bar, or arm, may be pulled across the top of the foliage in order to temporarily move it out of the way for the seeker cameras and/or the harvesting assembly to locate and grapple the target. In some examples, this foliage moving arm may be maneuvered parallel or substantially parallel to the top of the row bed, and pull across the top of the foliage, bending the plant, but not breaking the plant leaves. This may reveal targets under the foliage, those laying on the top of the row bed, or those caught up in the foliage.
In some examples, a flexible curtain may be dragged over the foliage, to avoid damage to the foliage, but still pull it out of the way for the seeker and/or harvester to operate. In some examples, this flexible curtain may be a plastic skirt, in some examples, it may be a fringed or sliced skirt. In some examples, it may have fringes that drape over the foliage, and yet flex around the foliage so as not to damage it. As the flexible skirt is pulled over the plants, it helps the seeker subassembly find the targets more easily by limiting the area to be targeted with a clean backdrop. The flexible skirt may be dragged from one side in one direction during a first harvest and the next time the other direction, to avoid biasing or pulling the foliage in the same direction each time.
In some examples, the overall traversing subassembly 152 may include a transfer conveyor 178. Such a conveyor may include any number of conveyor belts, chains, rope, or other mechanism that can pull materials from one place to another. Such transfer conveyor may be used to collect harvested targets and move them to a packaging subassembly, or storage unit.
In some example embodiments, the robotic arms 160 may be ruggedized in that the tolerances and durability of the arms are developed for outside, dirty employment. In such examples, the robotic arms are not to be operated in pristine factory settings. The systems described here should be able to operate in bad weather, precipitation, dirt, mud, heat, cold, and in jarring, rough conditions. As such, the bearings, tolerances, and actuators may be made of more durable materials than clean factory robotic assemblies. In some examples, extra gaskets may be fit into the various joints to keep dirt out of the more delicate metal couplings and pivoting features of the robotic arms. In such examples, gaskets may be made of rubber, plastic, or ceramic. The robotic arms may be made with fewer joints to minimize the number of potential problems that may occur. The robotic arms may be made of thicker materials, may be heavier, and be rust-proofed, waterproof, weatherized, and/or otherwise reinforced.
Robotic Arm Examples
In some examples, additionally or alternatively, the harvester subassembly as shown in FIG. 1 or any other embodiment, may include any number of robotic arms upon which the grappler, picker heads may be mounted.
FIG. 2 shows an example robotic arm picker assembly which may be configured to grapple targets from rows of plants as shown in the top down view 201, and from the side view 203. In FIG. 2, a picker head 202 is shown mounted to a robotic arm 260 which may position the picker head 202 in any of various poses in order to couple to a target 250. In the example, the targets are on a raised planter bed mound 201 which the robot arm 260 traverses from above in some examples mounted to a vehicle that traverses a row and targets and then grapples targets. In use, the system as shown in FIG. 2 may be positioned by the robotic arm 260, to harvest a target 250 by securing it using the steps as described in FIG. 3 as well as FIGS. 4A, 4B and 4C, then move the target 250, and/or deposit the target in storage as described.
In example embodiments, the robotic arm 260 may be any of the robotic assemblies described herein, and may include various numbers of joints thereby allowing for various degrees of freedom to move around and about the plants and rows, taking different angles. In some examples, the robotic arm 260 may include six degrees of freedom. In some examples, the robotic arm 260 may include four degrees of freedom, five degrees of freedom, six degrees of freedom, or seven degrees of freedom, or any other number. In FIG. 2, the first joint connecting the arm 260 to the harvester (not shown) may include a 360 degree rotating pivot joint 280, as well as a pivot hinge joint 282. The next joint is shown as a pivot hinge joint 284 connecting arm portions 260. The last joint on the example of FIG. 2 is the ones connecting the arm 260 to the picker head 202 as a pivot hinge joint 286. It should be noted that any kind of joints may be utilized in the robotic arm 260 including but not limited to pivot hinge joints, rotating pivot joints, ball and socket joints, condyloid joints, saddle joints, or any combination of these or other joints may be used. For example, the joint connecting the picker head 202 to robotic arm 260 may include multiple hinged joints or a ball and socket joint to allow the picker head 202 to turn, twist, and angle many multiple degrees in order to locate and harvest targets 250.
In various example embodiments, the robotic arm 260 may be any of various lengths, thereby affecting the range 295 of the arm, which may be tailored to the needs of the particular field or mound or target. In some examples, the robotic arm 260 may include one or more telescoping portions 288, which may be elongated and/or retracted, thereby affecting the length of that portion and the overall reach 295 of the robotic arm 260. Any of the portions of the robotic arm 260 could include such telescoping portions in any combination.
Picker Head Examples—Vacuum Point of Contact
In some example embodiments, the harvesting subassembly may include at least one picker head that first interacts with the target in the field to remove or detach the target from the plant it grows on. Such picker heads may be affixed to or be part of the robotic arms as discussed in FIG. 1 and FIG. 2. In some example embodiments, the at least one picker head may be mounted on or partially mounted on a robotic harvesting arm, alone or in combination with a camera and/or lighting system.
FIG. 3 shows an example picker head assembly, with a front 3A and side 3B view of the same assembly in detail. In the example shown, the main picker head assembly 302 is mounted on two actuators, one actuator for a pincers 304 and one actuator for an extender 306. In some examples, no extender actuator 306 is utilized. In examples where an extender is utilized, the extender actuator 306 moves in and down to move the main nozzle 303 up and down, relative to the robotic arm (not shown).
The main nozzle 303 may be a hollow tube which may be used to secure a coupling suction portion 330 to a target 350 such as a berry. In some examples, the coupling portion 330 may include one or more bellows or bellow configurations that allow the coupling portion to stay flexible and malleable to couple with the target 350. In some examples, the vacuum hose 303 may be connected with the main nozzle 330 to impart a suction or lower than ambient pressure within the nozzle tube 303, and thereby be able to attach to and secure a target 350. In some examples, a vacuum pump subsystem may be mounted on the harvesting subassembly and a vacuum hose may run through or around each harvesting picker robotic arm. In some examples, vacuum subassemblies may be mounted on the robotic arm itself, along with a vacuum hose on the picker head 302.
In some examples, the compression coupling portion 330 may be 1.250 inches in diameter, in some examples, the nozzle may be between 1.00 and 0.75 inches in diameter, but in any case, the nozzle could be customized to any size of intended target. In some examples, the compression coupling or bellows portion 330 may be 2 inches in diameter. In some examples, the compression coupling or bellows portion 330 may be between 1.75 and 2.25 inches in diameter. In some examples, the nozzle may have an effective ported area of 0.44 square inches. In some examples, the nozzle may have an effective ported area between 0.4 and 0.5 square inches. In some examples, the nozzle may have an effective ported area between 0.3 and 0.6 square inches. In some examples, the area of the compression coupling portion 330 may expand or stretch when attaching to a target berry and in such cases expand to an area of 1.56 square inches. In some examples, the area may expand to between 1.5 and 1.6 square inches. In some examples, the area may expand to between 1.4 and 1.7 square inches.
In some examples, the amount of suction power that the vacuum system imparts, may be 35 inches of negative vacuum. In some examples, 50 inches of negative vacuum may be used. In some examples, between 40 and 70 inches of negative vacuum may be used. Alternatively or additionally, in some examples, less than 80 inches negative of vacuum may be used so as to avoid damage to the target 350. In some examples, less than 68 inches of negative vacuum may be used. In some examples, the coupling or contact portion 330 of the bellows may spread the contact area over 20% to 30% of the target or berry so as to mitigate localized contact pressure to any one area and avoid damage, depending on the size of the target. Additionally or alternatively, the vacuum system may be able to reverse from suction to blowing air outward, to clear debris from the bellows, before switching back to a suction mode for harvesting.
The compression nozzle bellows portion 330 may include a malleable hood or coupling section 332 which may include one or more bellow sections, and a rim 334 around an opening 336 to aid in coupling to a target. In some examples, the compression coupling portion 330 is or made up of at least one of, or combination of a neoprene sleeve, a silicone sleeve, a rubber sleeve, or other natural or synthetic material that is soft and flexible including embodiments with embedded fibers and/or wires. Such a malleable coupling section 332 may be configured to deform or otherwise compress when a target 350 is contacted and may include baffles or other structure that allows for deformation and malleability. Such a deformation or compression may allow for the rim 334 to more easily conform to the target 350 and thereby form a better suction fit for the opening 336. In some examples, the bellows 330 portion may be removable and replaceable in the field using a friction fit and rim as described herein.
In some examples, the compression nozzle portion 330 may include an internal reverse conical mesh to help capture the target 350 yet be as gentle as possible on them as described herein. In such examples, the mesh creates an environment where the negative vacuum is acting on a broader surface of the target, thus minimizing the chance of target damage from localized contact to the grappler edges. This mesh forms a cradle for the target to lay in even while being picked, handled, and moved. Such a mesh can be made of silicone materials for durability and flexibility. Alternate materials may be used such as a wire mesh, a plastic mesh, or a combination of wire mesh with plastic coating. Silicon coating may be used on a wire mesh in some example embodiments as well. FIGS. 5, 6A-6C, 7A-7D, 8A-8C show more example embodiments that may be used herein.
In some examples, the compression nozzle portion 330 and the opening 334 may be sized for a most average target 350, big enough for the biggest targets and flexible, but able to grasp and vacuum even a smaller target.
Examples may also include an internal spring system, inside or integrated into the coupling portion 330. Such a spring system may be made of plastic or metal coil(s) that help return the coupling portion 330 back to an extended shape after a target is released by turning off the vacuum and thereby deposited. Additionally or alternately, an iris or camera lens feature may be integrated into the nozzle 330. In such examples, the system may be able to adjust the size of the opening or nozzle end for different sized targets, opening for larger targets, and constricting for smaller targets. In such examples, a coil or spring could be wound tighter for smaller targets and wound looser for larger targets.
Another portion of the example embodiment of FIG. 3 is the grappler spoons 312, 314. The grappler spoons 312, 314 may be configured with the main nozzle 303 between them and be configured to move in a pincer motion toward the nozzle 303 by a robotic actuator 316 and a hinge 318 arrangement. In some example embodiments, the grappler spoons include a cushion or pad 320, 322. In some examples, the cushion 320, 322 may be made of or include closed cell foam, neoprene, gel filled pads, liquid filled pads, open cell foam, layers of foam of different densities, a foam backing with a gel filled pad on top, and/or any combination of the above or other material that may cushion a target 350 when the grappler spoons 312, 314 pinch the target 350. In some examples, the material contacting the target 350 is no more than 20-30 durometer in hardness. More discussion of grappler spoon examples may be found in FIGS. 9, 10, 11, and FIG. 12.
In some examples, a picker head 302 may include multiple grappler spoons. In some examples, three spoons may be employed in a similar manner as those examples shown with two as in FIG. 3. In some examples, four grappler spoons may be configured in two axes around the picker head 302 assembly. In some examples, alternatively or additionally, the grappler spoons include a hinged and/or spring loaded portion at the end to better cushion the target 350 when pinched. In some examples, the grappler spoons 312, 314 may pivot about the nozzle 303 to impart a twisting motion to snap a berry or other stem as discussed herein using an actuator, screw drive, motor, or other feature to twist the picker head 302.
In some examples, a pneumatic trash cleaning air jet 324 may be mounted to the end of the grappler spoon 312, 314 in order to help clear debris. In such examples, air holes may be configured on the end lip of the spoons and face in various directions to direct air toward foliage. In some examples, a line of holes may be configured on the end lip of each grappler spoon 312, 314.
Example Picker Head and Picking Steps
As discussed in FIG. 3, the picker head may utilize features to grasp, secure, move, and then subsequently deposit an agricultural target such as a berry. Any combination of the features described here, alone or in combination may be used for this purpose.
FIGS. 4A, 4B and 4C show three snapshots in an example multi-step process of target acquisition and grappling using the picker head assembly 402 as described, where each step shows two angles 413, 405 of the same picker head assembly 402, front 413 and side 405. In an example target acquisition, first, 4A, the picker head 402 is directed to a target 450 by a seeker subassembly as discussed herein, using passed coordinates and/or manually steered. Once directed and in place, a robotic arm (not shown) maneuvers the picker head 402 into close proximity of the target 450 where the compression nozzle portion end 430 may attach to the target 450 (as described in FIG. 3) resting on the ground or surface 401 using low pressure imparted by the vacuum nozzle 403. In some examples, this vacuum nozzle 403 may be maneuvered in an extended configuration 444 using the extension actuator 406. In such a configuration, the compression coupling portion 430 may attach, suction, or otherwise temporarily hold the target 450 and thereby secure the target 450 with the vacuum suction through the vacuum tube 403.
Next, 4B, showing same picker head assembly 402, front 413 and side 405, the main nozzle tube 403 may be retracted using the actuator for the extender 406. In some examples, this is a generally upward motion 440 away from the ground 401 or surface and toward the interior of the picker head assembly 402. Some examples may forego the retraction step and not utilize the extension actuator 406 and/or may not be configured with one. In examples where retraction is utilized, with the target 450 attached to the compression nozzle portion 430 which is now retracted into the picker head assembly 402, the target 450 may be generally aligned with however many grappler spoons 412, 414 are fit on the picker head assembly 402. In some examples, the compression nozzle portion 430 and thereby the target 450 is retracted to align with the respective spoon cushion portions 420, 422 of the grappler spoons 412, 414, no matter how many grappler spoons are utilized.
In some examples, as shown through the FIGS. 4A, 4B and 4C, the spoon actuator 404 may be a piston (hydraulic or pneumatic) assembly in communication with a computer to receive instructions and send data and configured to raise and lower a bracket assembly 418 on the picker assembly 402 which may interact with a top end 472, 474 of each spoon arm 412, 414 to pivot each spoon arm 412, 414 about a pivot axis 410, 411, thereby opening and closing, or pinching the two spoons 420, 422 together, and moving them apart. In some examples, the spoon arms 412, 414 may include springs in the pivot axis areas 410, 411, which may bias the spoons in the closed or pinched position, and the bracket 418 may move in relation to the top spoon ends 472, 474 to move down to work against the spring tension to open the spoons, and up to allow the springs to pinch the spoons 412, 414 together. This actuation may take place due to the interaction between ramped or angled portions of the top of the spoon arms 472, 474, and the bracket 418 moving against the spring tension in the pivot axes 410, 411, pivoting each arm 412, 414 about its respective axes 410, 411.
As targets 450 may vary in size and shape, the alignment with the spoons 412, 414, may be obtained by retracting 440 the compression nozzle portion 430 so that the rim 434 of the compression nozzle portion 430 is at a place just above the respective cushion portions 420, 422 of the grappler spoons 412, 414 thereby ensuring that the respective cushion portions 420, 422 of the grappler spoons 412, 414 are able to pinch together to grasp 442 the target 450 without touching the compression nozzle portion 430 when they are in the closed position. It should be noted that more detail of example embodiments of the grappler spoon 412, 414, cushion portions 420, 422 is found in FIGS. 9, 10, 11, 12 and 13.
In some examples, sensors may be placed on or near the grappler spoons 412, 414 and/or cushions 420, 422, to sense the size and/or shape of the target 450. In such examples, a feedback loop may be used from the sensor data to adjust the distance the compression nozzle portion 430 is retracted to align the target 50 with the grappler spoons 412, 414. In some examples, such a sensor may be a light sensor, a laser sensor, a proximity sensor, piezoelectric pressure sensors, and/or a camera to align the target 450 with the grappler spoons 412, 414.
In some examples, a pneumatic trash cleaning air jet 424 may be mounted to the end of the grappler spoon 412, 414 in order to help clear debris. Such an air jet 424 may include one or more nozzles that are able to blast jets of air in various directions, thereby moving, flapping, or otherwise disturbing plants, leaves, dirt, sticks, stems, or other debris that the system is not trying to target, but might be in the way of a target. In some examples, the ends of the grappler spoons 412, 414 themselves may include one or more nozzles, ports, or holes for air jets to blast debris. In some examples, the outsides of the grappler spoons 412, 414, opposite the respective cushion portions 420, 422 may include one or more nozzles or ports, or holes for air jets to blast debris.
In the example of the third step, 4C showing the same picker head assembly 402, front 413 and side 405, when the compression nozzle portion 430 and thereby the suctioned target 450 is retracted 440 off the ground or surface 401 and aligned with the grappler spoon cushions 420, 422, the gripper spoons 412, 414 may be actuated by the grappler spoon actuator 404 and squeeze together 442 to grasp the target 450. In some examples, the retractable end of the baffler 432 may retract above or past the point where the pincher spoons 412, 414 may pinch together 442 so as to be able to hold a target 450 with the suction through the pliable baffles section 432 and the spoons 420, 422 at the same time. Only by retracting 440 far enough could a target 450 be secured by both systems at the same time. Such a configuration also allows for a handoff, for example, the suction may be turned off once the retracted portion 432 is secured by the pincher spoons 420, 422. More example of cushioned spoons may be utilized as described herein in FIGS. 9, 10, 12, and 13.
Thus, to secure a target 450, the vacuum suction nozzle 403 may include a pliable bellows 430 material that can conform to a target 450 and secure suction to it. The shape of the holes in the pliable bellows material, to apply the suction force may include various designs as discussed in FIGS. 5, 6A, 6B, 6C, 7A, 7B, 7C, 7D, 8A, 8B, and 8C.
In this third configuration, the target 450 may be grasped by the grappler spoons 412, 414, and may still be held by the compression nozzle portion 430 and the vacuum pressure from the main nozzle 403. In such examples, a feedback loop may be used from the sensor data to adjust the pressure used to squeeze the target 450. In some examples, additionally or alternatively, a single expansion spring (not shown) that may be connected between spoons 412 and 414 may be used. The spring may set the tension that the grappling spoons 412 and 414 may exert onto the target 450. In some examples, additionally or alternatively, the sensors may be piezoelectric pressure sensors on or under the grappler spoon cushions 420, 422. In some examples, additionally or alternatively, the sensors may include cameras to visually detect securing the target 450. In some examples, additionally or alternatively, the sensors may be tension sensors on the grappler spoon actuator 404 to sense the pressure exerted on the closure of the grappler spoons 412, 414. In some examples, additionally or alternatively, the sensors may be tension sensors on the gripper spoons 412, 414 and/or in a hinged portion of the gripper spoons 412, 414 to sense the pressure exerted on the closure of the grappler spoons 412, 414. In some examples, feedback loops may be analyzed by computer systems in communication with the grappler spoon sensors 412, 414, and also in communication with the actuators for the grappler spoons 412, 414 to adjust the pressure on the target 450, which may be used to secure differently sized targets 450 while minimizing damage to larger targets 450 and/or making sure smaller targets 450 are secured. It should be noted that further embodiments in addition to and/or in the alternative of the compression nozzle portion 430 are discussed in more detail in FIGS. 5, 6A, 6B, 6C, 7A, 7B, 7C, 7D, 8A, 8B, and 8C.
In some example embodiments, once the gripper spoons 412, 414 have secured the target 450, the vacuum suction may be turned off by the computer, reduced, or otherwise cut off from the compression nozzle portion 430 which would in turn release the pressure holding the target 450 to the compression nozzle portion 430 but leaving the target 450 in control of the grappler spoons 412, 414. In some examples, the padded grappler spoons 412, 414 may be configured to flick and turn the target 450 to remove them from their stems and thereby avoid having to cut a stem or plant in any way. This removal process of a target 450 from a stem may be advantageous in the shelf life of the target after harvesting and may be cheaper and easier to accomplish in the field.
In some examples, this twisting motion may be a 90 degree twist of the grappler picker head assembly 402. In some examples, this may be a 180 degree twist. In some examples, this may be between an 80 degree and 100 degree twist to snap a target 450 stem. In some examples, a snipping element may be used in lieu of or in addition to the snapping, twisting motion of the grappling spoons 412, 414. In such examples, a longer target stem may be desired, and snapping or twisting may remove the stem close to the target 450. Some example stem cutters are detailed in FIG. 14.
In some examples, the snapping of the stem by twisting may benefit if the stem of the target 450 plant is pulled out and away from the plant in order to impart a strain on the stem. In such examples, the pulling of the stem first, and then twisting the stem may result in cleaner and/or more accurate stem snaps. In some example embodiments, the extension/retraction actuator 406 may include a sensor that may sense resistance as the target 450 is retracted. In some examples, tension sensors may be placed in joints of the robotic arm(s) in communication with the computer systems, to make such a determination.
In some examples, the twisting motion may be imparted only when the resistance of the retraction of the target 450 meets a particular threshold, as determined by computer systems in communication with such sensors, thereby indicating that the stem of the target 450 is under strain or is otherwise stretched. Such resistance sensors may include a piezoelectric sensor, a strain gage, or other sensor mounted in or on the retraction actuator 406. In some examples, the target may be a berry that includes a calyx portion where a few leaves and the stem attach to the target berry. In some examples, this calyx portion may be identified by the seeker subassembly to help determine which direction to pull the target, normal to the calyx portion. In such examples, the camera and computer system may be able to identify a color variation between the berry itself and the calyx leaves and thereby the stem.
In snipping examples, a scissors, saw, clipper, or other sharp pincher may be secured to the picker head assembly 4C to cut the stem of the target 450 at a desired length. In some examples, one inch stems may be cut. Further additional or alternative examples may be found in FIG. 14. After securing the target by the steps described herein, the robotic arm assembly may then move the target, and deposit it by releasing the vacuum suction (if used) and opening the grappling spoons 412, 414, to release the target into a collection bin, conveyor belt, packing container, or other device to store, or transport the harvested targets.
Bellows Pliable Target Contact Examples
In some examples described herein, a pneumatic vacuum may be utilized to first secure the picker head assembly 402 to a target 450 such as a strawberry. This vacuum attachment 403 to a target 450 may allow for the picker assembly 402 to extract a target 450 from foliage, pick it off a stem and/or off a resting surface 401. Further vacuum attachment pliable bellows portions may be found in FIGS. 5, 6A, 6B, 6C, 7A, 7B, 7C, 7D, 8A, 8B, and 8C below. By securing a vacuum attachment 430 to a target 450 first, further even more grappling, twisting, and/or handling, may be accomplished with spoons 412, 414, as described herein to aid in harvesting and moving targets for example further spoon examples found in, FIGS. 9, 10, 11, and 12 below.
In such examples, pliable bellows 430 may be positioned on the robotic assembly to within range of a specified target 450 and the bellows section 430 extended 444 to within vacuum range. Through the bellows tube 403, a negative pressure may be generated by a pneumatic vacuum, pump, or other air or pneumatic suction device (not shown) thereby sucking air through the nozzle end 436, through the bellows tube 403 and thereby through the various ports, holes, slots, and/or other shaped voids in the end of the bellows as described in FIGS. 5, 6A, 6B, 6C, 7A, 7B, 7C, 7D, 8A, 8B, and 8C below. As such bellows 430 material may be made of pliable material to be able to better conform to the shape of a target more easily to thereby secure a better vacuum hold on the target.
Such a task is made more difficult by the variety of shapes, sizes, and orientation of targets needed to be grappled. Such a task is also made more difficult by the potential of nearby foliage, other targets, stems, dirt, sticks, etc. which could interfere with the vacuum suction, and/or reduce the vacuum pressure that may be applied to a target by the bellows. The purpose of the bellows vacuum system is to create enough of a vacuum attachment to the target such as a strawberry to secure the target from all or as many orientations as possible, including the apex point in examples of a strawberry. FIG. 5 shows an example of what such an interaction with a target X12 might look like from the inside or behind the bellows contact membrane 510. As shown, the target 550 is deforming the bellows contact membrane 510 due to the vacuum pressure being applied in the direction indicated 503 by a vacuum and tube (not shown). The various holes 508 shown cut or formed in the membrane 510 allow for the air to be sucked in the direction shown 503 by the vacuum pump, thereby pulling the target 550 toward the vacuum pressure. FIG. 5 shows the ability of the bellows membrane 510 to flex, stretch, bend, and conform its shape to the target 550. This is made possible by using a material that is both resilient to repeated use and stretching, yet pliable and soft enough so as not to damage or minimize the damage done to a target when the vacuum pressure is applied. Such materials may include but are not limited to silicone, plastic, rubber, polyurethane, polycarbonate, polyethylene, thermoplastic elastomers (TPE) such as but not limited to Thermolast K. The material used for bellows construction may pliable, stretchable, bendable, Food and Drug Administration compliant, heat resistant, sterilizable, and able to resiliently return to shape after being stretched. In some examples, the material used for the bellows may be of a durometer of 30-40 C with a finished thickness of 0.03 inches at the contact membrane 508 and an expansion rate of 500%. In some examples, the thickness at the contact membrane 508 may be between 0.02 and 0.04 inches. In some examples, the thickness at the contact membrane 508 may be between 0.01 and 0.05 inches. In some examples, the thickness at the contact membrane 508 may be between 0.025 and 0.035 inches thick. Any variation or range near these limits may be used. In addition, the base material may be food grade and/or Food and Drug Administration (FDA) approved.
As shown, once the target 550 is grappled, the vacuum flow 503 is obstructed or partially obstructed, and the vacuum negative pressure begins to rise, thereby causing the holes and/or slots in the contact membrane 508 to stretch open releasing more air flow to the outer diameter of the bellows 510. Depending on the size of the target, this may result in a cascading effect whereby the larger targets 550 stretch open the holes/slots in the contact membrane 508 to allow more attraction area onto the big targets 550 to offset their increased weight and mass over the small targets. Such a cascading effect may be most pronounced when the apex of the berry is engaged due to the pointy nature of a target apex (as shown in FIGS. 7B and 7D). This self-regulating suction port area adapts to the irregular shape of targets such as strawberries.
As can be seen from FIG. 5, the positioning and shape of the holes formed or cut into the bellows contact membrane may allow for gentle handling of the target 550 with maximum vacuum pressure applied to the target. Such a balance of enough pressure to grapple and handle the target, yet not enough to extract juice, bruise, damage, or otherwise harm the target may be made by configuring the holes in the membrane in a particular way so as to distribute the pressure evenly, allowing the membrane to flex, and allowing the bellows to grapple targets of various shapes, sizes, and orientations. In some examples, the bellows may spread the contact area over 20% to 30% of the target so as to mitigate localized contact pressure to any one area. Such a contact would thereby minimize damage to the target, yet maintain a sufficient suction hold on it. FIGS. 6A, 6B and 6C show example bellows construction cutaway views and FIGS. 8A, 8B and 8C are diagrams showing example bellows with different hole shapes and patterns to address these challenges.
It should be known that the material used to make the bellows section may include any of the below or other malleable, stretchable, and deformable yet resilient materials alone or in combination: silicone, plastic, rubber, polyurethane, polycarbonate, polyethylene, thermoplastic elastomers (TPE) such as but not limited to Thermolast K. Such material may be imparted with fibers to make them stronger and/or more resilient. Such materials may be doped with chemicals, vulcanized, sprayed or treated with chemicals to help with pliability and to prevent cracking or breakdown. Such material may be food grade material as approved by the Food and Drug Administration to be able to contact food and for food handling. Any of the descriptions of the bellows, contact coupling sections, or other target contact elements may include any or all, and any combination of these materials, in the shapes, sizes, and features described here and are not intended to be limited.
Turning now to the size, shape, and configuration of the bellows contact membrane, FIGS. 6A, 6B and 6C are diagrams showing example cutaways of bellows 612 construction as described in the embodiments disclosed herein. In FIG. 6A, a cutaway view of the bellows 612 including the contact membrane 602 are shown. The vacuum tube 610 is shown with the bellows section 612 secured to it and/or around the end of it. In the example, the bellows section 612 is friction fit and secured by a rim or matting flare 614 on the more rigid vacuum tube 610 which interacts with the more malleable bellows material 612. The bellows section 612 may be configured to attach to or around the vacuum tube 610 and be removable, such that it could be quickly and easily replaced in the field if it is damaged or worn out. Any of the bellows, contact membranes, etc., described herein may be configured with such a removable feature set allowing them to be replaced, removed, cleaned, etc. in the field or maintenance depot.
In use, the vacuum pump (not shown) may be configured to impart a suction in the direction shown 603 which would thereby pull air through the holes in the contact membrane 602 as described herein.
The thickness of the material making the bellows section 612 may vary in different sections of the bellows 612. The thickness of this material may influence how much the material may stretch, deform, or otherwise shape around a target in use as shown in FIG. 5. For example, the material into which the holes are cut or formed in the contact membrane 602 may be between 0.02 and 0.04 inches. In some examples, it may be between 0.01 and 0.05 inches thick. In some examples, it may be between 0.02 and 0.06 inches thick. The rim 620 around the contact section 602 may be between 0.03 and 0.04 inches thick. In some examples, the thickness around the contact section 602 may be between 0.02 and 0.05 inches thick. In some examples, around the contact section 602 may be between 0.03 inches and 0.06 inches thick. In some examples, the rim 622 around the vacuum tube 610 may be between 0.125 and 0.187 inches thick. In some examples, the rim 622 around the vacuum tube 610 may be between 0.1 and 0.2 inches thick. In some examples, the rim 622 around the vacuum tube 610 may be between 0.1 and 0.3 inches thick.
FIG. 6B shows another cutaway example of a bellows section 632 with contact surface section 634. This is example, the thickness of the sidewalls of the bellows section 636 is thicker than that shown in FIG. 6A. In FIG. 6B, the sidewalls 636 are between 0.125 inches and 0.187 inches thick near where it encircles the vacuum tube 611. In some examples, the sidewalls 636 are between 0.1 inches and 0.2 inches thick near where it encircles the vacuum tube 611. In some examples, the sidewalls 636 tapers to a thickness of between 0.05 inches and 0.06 inches at the rim 638 of the ring around the contact membrane 634 with holes to provide the suction pressure to a target (not shown). In some examples, the sidewalls 636 tapers to a thickness of between 0.04 inches and 0.07 inches at the rim 638 of the ring around the contact membrane 634. In some examples, the sidewalls 636 tapers to a thickness of between 0.03 inches and 0.07 inches at the rim 638 of the ring around the contact membrane 634. In the example of FIG. 6B, the thickness of the outer rim 638 becomes thinner near the front of the rim 639 closer to the contact membrane 634. Such examples could include thicknesses of between 0.04 inches and 0.05 inches at the outer portion of the rim 638 and between 0.03 inches and 0.04 inches at the front portion of the rim 639. In some examples, the thicknesses may be between 0.03 inches and 0.06 inches at the outer portion of the rim 638 and between 0.02 inches and 0.05 inches at the front portion of the rim 639. In some examples, the thicknesses may be between 0.02 inches and 0.07 inches at the outer portion of the rim 638 and between 0.01 inches and 0.07 inches at the front portion of the rim 639. Such thinner material at the front portion 639 may allow for grappling more delicate targets, smaller targets, or lighter targets.
FIG. 6C shows another cutaway view of a bellows 642 but in this example, the outer rim 648 and front portion 649 of the rim around the contact membrane 644 is thicker, between 0.03 and 0.04 inches thick. In some examples, the front portion 649 of the rim around the contact membrane 644 may be between 0.02 and 0.07 inches thick. In some examples, the front portion 649 of the rim around the contact membrane 644 may be between 0.02 and 0.1 inches thick. The different thickness in the material for the bellows 632 may allow for a more robust and/or longer work life for the bellows section which may be replaced if damaged or worn out, as described herein.
FIGS. 7A, 7B, 7C, and 7D are diagrams showing example bellows interacting with differently sized targets as described in the embodiments disclosed herein. In FIG. 7A, both a cutaway view 712 and a perspective view 722 of a medium sized target 750 interacting with the bellows with the vacuum pressure 703 pulling the target 750 toward the bellows 712, 722. In the example, the target 750 is medium sized and fits within the overall main section of the contact membrane 702 when grappled from the side. The outer rim of the bellows 704 does not come into contact with the target 750 just because of the size of the target 750 compared to the bellows 712 width.
In FIG. 7B, the cutaway view 732 of the bellows shows the target 751 contacted at the apex 752 point portion by the contact membrane 732. As in FIG. 5, the contact membrane 732 is shown deformed or stretched to secure the target 751 and hold it due to the vacuum pressure 703. In the example, the rim 734 does not contact the target 751 but surrounds it because of the size and shape. The perspective view 742 shows the same target 751 secured in the contact membrane 732 surrounded by the rim 724.
FIG. 7C shows a cutaway of the bellows 756 and a perspective of the bellows 758 grappling a larger target 752. Because the example target is large, both the contact membrane 760 and rim 762 contact the side of the target 752 and draw it toward the bellows 756, 758 toward the direction 703 of the vacuum pressure through holes in the contact membrane 760. This figure demonstrates how even larger targets may be grappled by the bellows 756, 758 and still maintain a good vacuum seal on the target 752 to harvest and move it, as described herein.
In FIG. 7D, the cutaway view 772 of the bellows shows the target 753 contacted at the apex 754 point portion by the contact membrane 774. As in FIG. 5, the contact membrane 774 is shown deformed or stretched to secure the target 753 and hold it due to the vacuum pressure 703. In the example, the rim 776 contacts the edges of target 753 and surrounds it because of the size and shape. The perspective view 782 shows the same target 753 secured in the contact membrane 774 surrounded by the rim 776. In this figure, it can be seen how even larger targets 753 may be grappled, harvested and moved, from various orientations including the apex 754.
FIG. 8A shows two examples 802, 820 of the end of a bellows unit, the contact membrane where the holes are made in the pliable material, allowing the air to flow through and thereby suction pressure to be generated on a target as shown in FIG. 5. The end-on examples of the contact membrane 804, show various holes and slots in the pliable material which may effectuate different pressures on a target when applied with a vacuum pressure.
The first example 802 in FIG. 8A shows the main bellows contact membrane circular shape 804 along with the interior rim 806 of the bellows section. Between the outside 804 and the interior rim 806, the bellows contact membrane have no holes or other air ports. This section between outside 804 and the interior rim 806 may be configured as an annular rim which may cushion and secure to the target when applied in use, and may be any of various materials and/or thicknesses as described in more detail in FIGS. 6A, 6B and 6C. Inside this rim between 804 and 806 is the section with the patterned holes allowing air to flow in use. In the first example of 802, the membrane includes annular positioned slots 812 ringing the interior of the membrane in generally concentric circle shapes. Regarding all of the contact membranes of the drawings here, in use, the initial contact area of the target or berry covers over the innermost holes or parts. At that time, the vacuum flow is at least partially obstructed and the vacuum negative pressure begins to rise, thereby causing the circular slots to stretch open allowing more air flow to the outer diameter of the bellow rings. Such a situation may have a cascading effect on larger targets whereby the larger target berries stretch open the circular slots more and thereby allow more attraction area and airflow which may allow for harvesting targets with increased weight and mass. In such examples, smaller target berries may not stretch the contact surface as much as the larger targets, and thereby not increase the air flow as much or the contact surface, which is unnecessary due to smaller targets having smaller mass and weight.
In some examples, the cascading effect may be most pronounced when an apex, point, or tip of a target berry is engaged as shown below in FIGS. 7B and 7D. This self-regulating suction port area lends itself well to adapting to the irregular shape of target berries. In the various examples of contact membranes and shapes of holes and slots therein, any of various arrangements may be made to aid in contacting, attracting by negative suction power, and holding, yet not damaging the target. The examples listed here may be combined in any way and may be configured in any combination of the embodiments listed or with other combination elements.
The slots 812 are elongated between 0.12 inches and 0.14 inches in length and between 0.04 inches and 0.05 inches wide. In some examples, the slots 812 are elongated between 0.1 inches and 0.2 inches in length and between 0.02 inches and 0.08 inches wide. The example shows multiple annular rings of slots, four in this example, but could be any number with the most interior ring 814 shown toward the middle of the membrane 802. In the example, the rings of increasingly smaller slots between 812 and 814 include approximately the same sized and shaped slots, with generally rounded ends and a generally consistent thickness, except for the decreasing diameter of the overall ring shape they make, and are overlapping in their positioning. That is to say, the most outside ring of slots 812 each ends in an approximate mid-point for the next interior ring of slots, and then again overlapping each ring until the most interior ring 812. The number of slots on each ring 812, 814 may depend on the size and shape of each slot and the spacing between rings may vary between 0.04 and 0.06 inches. In some examples, the spacing between rings may vary between 0.02 and 0.09 inches. In some examples, the spacing between rings may vary between 0.02 and 0.125 inches.
In the example of FIG. 8A, inside the rings of slots 812 to 814, a center most section includes slots that are not annularly placed around the middle as the rings 812 are, but slots 817, 816 angled toward the center of the membrane 804 with a central hole 822. The sizes and shapes of these most central slots 817, 816 may vary between 0.03 and 0.06 inches in width. In some examples, the sizes and shapes of these most central slots 817, 816 may vary between 0.02 and 0.08 inches in width. In some examples, alternating centrally points slots 817 are longer than others 816. In some examples, the longer slots 817 may be between 0.2 and 0.21 inches long, and 0.03 and 0.06 inches wide. In some examples, the longer slots 817 may be between 0.1 and 0.3 inches long, and 0.02 and 0.09 inches wide.
In some examples, the centrally pointing slots 817, 816 may be pie shaped or wedge shaped with the most exterior portions wider than the more interior portions. In some examples, the centrally pointing slots may be between 0.08 and 0.09 inches wide at the exterior portion and between 0.04 and 0.05 inches wide at the end closest to the center. In some examples, the centrally pointing slots may be between 0.04 and 0.125 inches wide at the exterior portion and between 0.02 and 0.09 inches wide at the end closest to the center. In some examples, the centermost hole 822 is circularly shaped and 0.12 inches in diameter. In some examples, the centermost hole 822 is circularly shaped and between 0.1 inches and 0.2 inches in diameter.
The next example of hole patterns in the bellows contact membrane 820 includes similar annularly placed slots 824 in concentric circle shapes, similar to the design in 802 but the interior holes are shown as circularly shaped 826 instead of elongated slots. In the example of 820 the interior of the contact membrane includes two concentric rings of circularly shaped holes 826 and one central hole 828. In some examples, the diameters of the concentric ring circles 826 may be between 0.09 and 0.1 inches. In some examples, the diameters of the concentric ring circles 826 may be between 0.05 and 0.2 inches. In some examples, the central hole 828 may be between 0.12 and 0.13 inches in diameter. In some examples, the central hole 828 may be between 0.07 and 0.2 inches in diameter.
FIG. 8B shows two more examples of contact membranes 840 and 860. In 840 example the concentric ring of slots 842 may be overlapping and include dimensions of between 0.13 and 0.14 inches long, between 0.03 and 0.05 inches wide and have a spacing between rings or between 0.06 and 0.075 inches. In some examples, 840 example concentric ring of slots 842 may be overlapping and include dimensions of between 0.08 and 0.2 inches long, between 0.1 and 0.09 inches wide and have a spacing between rings or between 0.02 and 0.125 inches. The example shows the overlapping pattern shown above where the midpoint of one slot approximately lines up with the end or the spacing between slots in the next ring inward or outward, whichever pertains. In the example of 840, the interior concentric ring holes are oval shaped 844 with the apex pointing toward the central hole 846. In some examples, the outer ring 848 of the inner oval shapes is thicker with a width between 0.1 inches and 0.12 inches. In some examples, the outer ring 848 of the inner oval shapes may have a width between 0.05 inches and 0.2 inches. In some examples, the inner of the two oval rings 848 includes ovals with thinner widths between 0.08 and 0.1 inches. In some examples, the inner of the two oval rings 848 includes ovals have widths between 0.05 and 0.25 inches. In some examples, the oval rings may have their apexes pointed around the ring instead of toward the center hole 846 as shown. In some examples, the thicker ovals may be positioned in the interior of the two oval rings and the thinner width ovals may be positioned on the outer of the two inner oval rings. In some examples, the number of rings may be one, two, three, four, five, or more rings of holes.
In the example 860, the concentric ring of slots 862 includes slots with wider widths than 840. In some examples, the widths of slots 862 is between 0.05 and 0.07 inches wide. In some examples, the widths of slots 862 may be between 0.02 and 0.09 inches wide. In some examples, the number of concentric rings of slots is three as shown. In some examples, the number of concentric rings slots is two, one, four, or more (not shown). In some examples, the concentric rings of ovals 864, 868 may be between 0.02 and 0.09 inches wide. In some examples, the central hole 866 is between 0.125 inches and 0.014 inches in diameter.
FIG. 8C shows another embodiment of a contact membrane 870 of the bellows with the outer rim 872, but the example shows the concentric rings of slots 874, 876, etc. are not shaped as those in FIGS. 8A and 8B with generally rounded ends and a generally consistent thickness, instead, the thickness varies to make the slots more thick on one side than the other as in 874 and in some examples, even include a side with a pointy, triangular shape, bean shape, or other 877. In some examples, the ring of concentric slots may be similarly shaped to those in FIGS. 8A and 8B with a generally consistent thickness 878. Any combination of these shapes or other shapes may be configured as described here.
Spoon Examples
As shown in FIG. 3, the cushion or spoon portions 320, 322, of the gripper pincher arms 312, 314 may include different embodiments and combinations in order to help secure yet cushion the agricultural target for harvesting. Therefore, such cushions may be firm yet flexible, made of FDA compliant material for food handling, yet rugged for outdoor use.
And because the cushions or spoons may need to be flexible to be soft, they may wear out or get damaged. They may become dirty or spoiled. In some examples below, the spoons may include pads or membranes that may allow for removal of the pads for cleaning or fixing, or replacement of pad material for different targets. Below are some examples, alone or in combination, which may be used as cushions, spoons, pads, or otherwise agricultural target grapplers for the harvester.
FIG. 9 is a diagram showing example spoon grappler construction as described in the embodiments disclosed herein. As shown in FIG. 9, one of potentially multiple spoon portions is shown attached to a flange 910, which in turn would be attached to the grapplers and actuators as shown in FIG. 3 etc. The spoon portion may include a rigid outer rim 902 and a rigid inner rim 904 attached to the flange 910. In some examples, the two rims 902, 904 are separated from one another 906 with a gap or space 920 between the two. The target to be grappled would be captured within this gap 920, cushioned by the material covering the outer rim 902 and surrounding the padding 922 of the inner rim 904.
The two rims 902, 904 may each be wrapped in or otherwise covered include soft, pliable material that would not damage the targets. In some examples, the outer rim 902 may be completely covered with a plastic, neoprene, or other soft and deformable yet resilient and FDA compliant material that would cushion the target. The inner rim 904 may include a soft and pliable rim section 922 similar to an earphone with padding and soft yet resilient cushioning. Together, the material covering the outer rim 902 and the padding around the inner rim 904 may work together to gently yet firmly attach to a target, and squeeze against another spoon arrangement, configured as a mirror image to that shown in FIG. 9, to hold a target, along with or in conjunction with the vacuum and bellows assembly as described herein.
FIG. 10 is a diagram showing more example spoon grappler construction as described in the embodiments disclosed herein including a top down view 1030 and a front view 1040. In the example, the outer rim 1002 and inner rim 1004 of the spoon section are shown attached to the flange 1010. The front view shows the inner rim padding 1022 and the interior space 1020 formed inside the padding of the inner rim 1022 portion like an earphone configuration. The target would be captured between the inner rim 1004 padding 1022 cushioned by material covering the outer rim 1002, and a second spoon assembly (not shown) which could squeeze together gently yet firmly enough to secure a target.
FIG. 11 is a diagram showing example spoon grappler cutaway construction as described in the embodiments disclosed herein. In the example of FIG. 11, the flange 1110 is attached to only one rim 1102 upon which a resilient yet pliable membrane 1104 is attached. The membrane 1104 includes a thin section 1122 which may contact the target when assembled. The rear of the assembly 1130 may be made of a firmer or harder material than the membrane 1104 and would serve as a structural support for the rim 1102. In some examples, the membrane 1104 would be able to be replaced by a friction fit around the rim 1102 would be able to be replaced in the field in order to repair damaged membranes 1104 or change them due to weather conditions, or targets. In some examples, the membrane material 1104 may be any of those described herein for the bellows, or spoon membranes in any combination.
FIG. 12 is a diagram showing an example spoon grappler interacting with a target as described in the embodiments disclosed herein. FIG. 12 shows an end on view 1230, a frontal view 1240 and a perspective view 1242 of a grappler spoon with a flat membrane embodiment. The flange 1210 attaching the spoon to the picker head assembly (not shown) is shown attached to a rim section 1204 covered by a membrane 1222. The spoon is shown grappling or interacting with a target 1250. The spoon in use, would pinch the target 1250 between itself and at least a second spoon (not shown) and/or the vacuum bellows arrangement to hold and harvest a target. As the membrane 1222 is made of soft, pliable, yet resilient material, it may deflect when grappling the target 1250 to cushion it softly, yet firmly enough to hold it while snapping the stem or otherwise removing it from the plant (not shown).
FIG. 13 also shows a perspective 1342, end on 1330 and frontal 1340 view of a spoon embodiment. FIG. 13 shows a similar arrangement as FIG. 12 but with a larger target 1350 interacting with the membrane 1322 of the spoon on the rim 1304 and flange 1310. As can be seen from FIG. 13, even a larger target 1350 is able to be grasped by the spoon rim 1304 and membrane 1322 to secure it.
The material used for membranes for the spoons in FIGS. 9, 10, 11, 12, and/or 13 may be similar to, or the same as those for the bellows, for example, they may be pliable, stretchable, bendable, Food and Drug Administration compliant, heat resistant, sterilizable, and able to resiliently return to shape after being stretched. In some examples, the material used for the membranes of the spoons may be of a durometer of 30-40 C with a finished thickness of 0.03 inches at the contact membrane and an expansion rate of 500%. In some examples, the thickness at the contact membrane may be between 0.02 and 0.04 inches. In some examples, the thickness at the contact membrane may be between 0.01 and 0.05 inches. In some examples, the thickness at the contact membrane may be between 0.025 and 0.035 inches thick. Any variation or range near these limits may be used. In addition, the base material may be food grade and/or Food and Drug Administration (FDA) approved.
Stem Cutter Examples
FIG. 14 shows an example of a stem cutter embodiment 1402 which may be used in conjunction with a picker head assembly 1420. In the example, the cutter assembly 1402 may slide up and down 1430 relative to and outside the picker head assembly 1420 as a sleeve, wherein the vacuum bellows portion may extend through 1440 the stem cutter assembly 1420 to attach to a target, then retract 1442 until the stem cutter 1402 contacts the stem to cut it.
The example stem cutter shows a rotating 1412 motion for the stem cutter 1402 to cut or shear the stem with saw teeth 1422. In the example, the stem cutter 1402 may include a nested or second cutter 1404 which may rotate counter to one another to impart a shearing force on a stem (not shown).
The spinning or turning of the stem cutter assembly 1402 and/or 1404 may be imparted using a rotating motor assembly (not shown) that may attach to the stem cutter sleeves 1402, 1404 to spin them 1412. Such spinning may not need to be a full 360 degree rotation, but in some examples, may be 10 degrees, 30 degrees, 60 degrees, or 180 degrees. In some examples, a spinning of the inner cutting sleeve 1404 and the outer cutting sleeve 1402 in opposite directions would be enough to slice, cut or otherwise sever a stem from a target attached to the vacuum bellows 1440 when placed in contact with one another. Such spinning may be accomplished and controlled by a computer system programmed to slice stems when the picker head assembly pulls the target from its plant. In some examples, as described herein, a tension sensor may be utilized on the robotic arm assembly, in a joint, to sense that the target is being pulled and resisting being pulled from the plant, and at a predetermined tension threshold, activate the spinning stem cutters 1402, and./or 1404 to cut the stem.
Seeker/Sensor Subassemblies
In some examples, the harvesting described herein is directed by a seeker subassembly that is able to identify targets for harvesting, pass coordinates for the targets to the picker subassembly for extraction. Such seeker subassemblies may include any number of cameras (visible light, thermal, UV or other), radars, lidars, lasers, acoustic location finders, GPS, inertial navigation systems, piezoelectric sensors, and/or any combination of these or other sensors to locate and identify targets.
In some examples, a seeker subassembly vehicle works independently from the harvester subassembly, and in some examples, the two subassemblies are on the same traversing machine. In some examples, the harvesting subassembly has its own wheels and/or tracks or combination of both, to traverse down a row of agriculture and harvest the mapped targets it receives from the seeker subassembly. In some examples, the seeker subassembly vehicle and harvesting subassembly vehicle are able to mate, connect, and/or otherwise work in concert by connection. In some examples, this connection includes a wired connection to allow for target information to be passed from the seeker subassembly to the harvesting subassembly. In some examples, the two subassemblies communicate wirelessly. In some examples, a combination of wired and wireless communication may be arranged.
In some examples, either or both the subassemblies may include location sensing and determining devices. In some examples, GPS location sensors may be configured on both or either subassemblies for the computer systems to determine locations and/or steer. In some examples, cellular towers and signals may be used for location sensing by the subassemblies. In some examples, inertial navigations systems may be used such as a ring laser gyro, a magnetic gyro and/or any other combination of such with a computer system. In some examples, additionally or alternatively, the cameras in the seeker subassembly and/or other cameras on the harvesting subassembly may be used to identify and track an agricultural row down which the vehicle may be steered. The location sensing and/or steering may be fed into any computer system, either located on the harvesting/seeking systems or remotely, in order to autonomously, semi-autonomously and/or allow for human activated remote steering. Any combination of these or other systems may be used to locate and/or steer the systems here.
The initial configuration is intended to utilize self-steering on row with the intention to utilize a droid tender (person) to steer the droid off-row for unloading accumulated berry containers and reloading empty containers, then finally steering the droid back onto a new row to be picked. The droid will have the ability to be converted to full autonomous mode for turnaround at the head lands as well as unloading and loading berry containers.
In some examples, the seeker subassembly may include one or more robotic arms with sensor(s) attached. In such examples, the robotic arms may include joints which provide multiple degrees of freedom of movement. Such multiple degrees of freedom may be useful to move in, on, around, and through foliage of target agriculture to find the target to be harvested, for example, a berry within the foliage of a shrub or plant.
In some example embodiments, additionally or alternatively, the seeker subassembly robotic arms may include at least one camera as described herein. In some examples, the seeker subassembly robotic arms may include at least one light system as described herein. In some example embodiments, a single robotic arm may include a multitude of cameras and light systems. In some example embodiments, additionally or alternatively, the cameras and/or lights may be integrate into the harvesting robotic arms. For example, referring to FIG. 1, in some examples, the picker 102 on the end of the robotic arm 160 could include a camera and light system. In some examples, the sensors and/or light systems may be attached to the frame 153 of the system.
In some examples, a cascade of hierarchal cameras may be employed on the systems. In such examples, larger scope or angled cameras may be used to identify one or more targets from a wide angle. In such examples, a first coordinate mapping may be calculated using the wide angle lens cameras. In such systems, the back end computers may receive the first coordinate or mapped information and use that to focus a second camera system on the selected targets for a more refined targeting. The second set of narrower angle cameras may be configured to hone in on the targets that the wide angle system first mapped, and refine or detail a tighter set of coordinates for each target. This arrangement of passing from wide angle camera systems to a second set of narrower camera systems may allow for a tight control loop for the picker assemblies.
In some examples, one or more laser sensors may be configured on the systems to find, locate, and map targets. In some examples, lasers may be employed to augment other camera assemblies to illuminate targets for cameras with matched wavelength receptors to capture images. In some examples, lasers may be used exclusively. In some examples, each picker head may include its own laser system to be used as a range finder, a color differentiator, and/or other sensor for the final picking action at the target itself.
In some example embodiments, additionally or alternatively, the seeker subassembly and/or harvesting subassembly robotic arms include at least one foliage moving flipper or pneumatic air jets configured to alter, move, displace, and or otherwise gently maneuver the foliage of the plant to better expose the berries.
In some example embodiments, the seeker subassembly may include a camera and/or multiple cameras arranged so as to be able to view the target foliage and thereby the target agriculture to be harvested. In some examples, multiple cameras may be arranged on the seeker subassembly such that images taken from the multiple cameras may be processed by a computer system to create three dimensional (3-D) images using machine vision. In some examples, these images are made of pixels and the computer systems is able to identify targets represented by pixels to be harvested and map the targets in three dimensions. In some examples, the cameras may be configured to acquire multi-spectral or hyper-spectral imagery to enable the use of advanced analysis algorithms for evaluating fruit health, quality and ripeness. In some examples, the images gathered may include those of a thermal imaging system for evaluating the temperature of the berry to be harvested. These cameras may comprise of cooled or uncooled sensors generating area-scanned images of at least 640×480 pixels. Some embodiments may utilize a single thermopile based sensor to provide an integrated temperature measurement of the mean temperature of the target such as a berry.
To deal with leaves, stems and trash obstructing access to the targeted berry, the berry grappler may include the ability to recognize obstructions and perform a second operation to clear access to the targeted berry. The berry grappler may be equipped with a trash diverter located at its most forward tip of the grappler spoons. The trash diverter may be equipped with air jets, or foliage hook device, or rotating paddle device and other appropriate methods to displace the obstruction. In operation, the berry grappler trash diverter may move in a progressive diameter rotation around the targeted berry location, clearing the obstructions. This second operation may only take place when obstructions are viewed by the stereo cameras.
In some examples, the three dimensional image data processed by and sent from the camera(s) may allow for a virtual reality environment to be created for a human user. In such examples, a virtual reality headset or display may be utilized by a user, remote or close to the harvester, to identify target agriculture and thereby send the target mapping coordinates to the harvesting machine for harvesting.
In some examples, data created by the cameras and data created by the human selection of agriculture may be stored by a computer device. In such examples, the identification data may be amassed in order to analyze and later create algorithms for neural network engines to process. In such examples, after much data of targeting, identification, and harvesting information is gathered, an neural network engine can be trained may be able to replicate some or all of the human targeting using the three dimensional maps.
Examples of cameras which may be used in the described systems include stereo vision with resolution of 1920×1080 and frame rates of 30 per second. Some examples include stereo vision with resolution of between 1500-2000× between 1000-1200 and frame rates between 10 and 50 per second.
Lighting Examples
In some example embodiments, the seeker subassembly includes various specialized lighting which may be used to find and identify targets. Such lights may be configured on the ends of robotic arms, integrated into robotic arms that include picker heads, or cameras. Examples are shown in FIG. 1. Such lights may be fixed onto other sub-assemblies on the seeker assembly and/or harvesting sub-assembly.
In some examples, such specialized lighting may be configured to emit a certain wavelength or spectrum of wavelengths such as but not limited to visible light, infra-red light, and/or ultra-violet light. In some examples, the lighting may be at a wavelength that excites items to fluoresce. In some example embodiments, light spectrum filters may be used by the cameras described herein to filter out or delete wave lengths of light that would otherwise block out any fluorescent properties reflected or emitted by targets such as berries.
In some examples, the specialized lighting may be light emitting diodes which are tuned to emit light at a specific frequency. In some examples, that frequency may be a combination of 470 nm (blue) and 635 nm (red). In some examples, the lights may be LED lights. In some examples, the lights may be incandescent lights. In some examples, the lights may be halogen lights, fluorescent lights, metal-halide, neon, high-intensity discharge lamps, or any permutation or combination of any of the above.
Mapping and Passing Target Coordinates
In some example embodiments, additionally or alternatively, sensors onboard the harvesting systems such as machine vision camera and computer systems may be used to map target agriculture in three dimensions and pass the coordinates to the harvester subassembly for harvesting. These mapping coordinates may be described in a global coordinate system such as Universal Transverse Mercader (UTM), or a local coordinate system frame relative to the coordinate system defined by the three dimensional imaging system on the harvester. In some examples, an X,Y,Z coordinate system may be employed using an anchor point in the camera view and/or on the traversing machine itself.
The various sensors described herein including but not limited to visible light cameras, infrared cameras, ultraviolet cameras, lidars, radars, lasers, or other sensors may be used to scan the produce plants and identify targets. Using the automated, semi-automated, or manual selection processes and systems described herein, the systems could generate coordinates for selected targets. These mapped target coordinates may then be queued in a buffer or database, for the harvester subassembly to harvest. In some examples, after one coordinate is added to the harvesting coordinate queue, more targets may be added to the queue to be harvested in turn. In such examples, the targeting subassembly, machine vision, and target mapping may occur without lag or delay in the handoff from targeting to harvesting, and not be hampered by the limitations of the harvesting subassembly itself. In such a way, in some examples additionally or alternatively, the targeting subassembly may be mounted on a separate vehicle to travel at its own speed and send targeting mapped data to the harvesting subassembly by wireless communications. In some examples, the targeting subassembly may be a part of the overall machine and connected to or in communication with the harvesting subassembly to pass the targeting mapped coordinate queue by wired communications to the harvesting subassembly. In some examples, a cloud or distributed computing resource may be utilized so that the targeting queue may be relayed or sent to the harvesting subassembly wirelessly.
In some examples, the mapping may be done early or before a harvester machine may come down a row. Additionally or alternatively, in some examples, mapping may be done just before harvesting, on the same machine in some examples to minimize the variables of the berries and/or foliage moving. Any time between target mapping and harvesting may be utilized, depending on the circumstances of the harvest.
In some examples, mapping information may be stored in a remote server, cloud server, or distributed system, for the purpose of future analysis (post processing) of the imagery to evaluate the condition of the plant. Post processing operations may include an evaluation of the plant for disease, nutrient deficiency, unripe berry inventory, and/or other plant damage. Data gathering and analysis on all types of agricultural specifics may be accomplished using the suite of cameras and/or sensors on the systems described herein. For example, outputs of post processing operations may be utilized to selectively address in-field issues at a plant-local scale that may otherwise require broad remedies using traditional methods. Other outputs of post processing operations may generate statistical data related to observations and measurements that are made while the harvester is operating in the field that can be advantageous to the growers business efforts.
Automation and Remote Examples
Additionally or alternatively, the systems described here may be used to harvest agricultural targets in an automated, semi-automated, or even manually controlled manner. In some examples, the semi-automated manner may be arranged in a remote setting, allowing for a human to interact with camera views from the harvester to help target the produce.
The variations on these options depend on how much a remote or local computing system may be programmed to identify and harvest a target. For example, in a fully manually controlled system, a human operator may control the movements of both the seeker system and the harvesting system. In such examples, by remote control using a joystick or other computer driven operating device(s) a human could scan the rows of plants for a target using the camera systems, and even maneuver the robotic arms that the camera systems are connected to, to identify targets, and then use a control system such as a joystick to maneuver the picker head assembly to the target, and then harvest the target as described herein. Such examples would allow for remote operation of the systems such as by wireless control to allow for human controllers to be stationed anywhere in the world, through some kind of wireless uplink.
The other extreme of control systems would be a fully automated system. In such a system, the traversing machines would move down a row of agricultural targets and the seeker subassembly would use machine learning/artificial intelligence/neural networks/and/or other programming to seek out and identify targets with the seeker subassemblies and then harvest them as described using the picker heads. Such examples would depend on computer algorithms and programs to determine using the inputs from the cameras and sensors, what a target may be and where they are located. For example, a color camera may be used by the computer system to detect a red strawberry amongst the green foliage of the plant it is growing on. Then a laser system could be used to determine a proximate location and range from the system and the computers could use that information to triangulate a three dimensional coordinate system and identify where the target is located in space, relative to the traversing machine. Next, the coordinates could be passed to the harvesting subassembly where the picker heads may attach to and harvest the target strawberry, in some examples using its own sensors such as cameras and lasers.
The middle-ground option between the fully automated and the manually controlled system would be some variant of semi-automated seeking and harvesting. The degree of semi-autonomy and which portions were automated and which manually controlled could vary from separate subassemblies. For example, the seeker subassembly may be more manually controlled with a human interacting with the cameras and sensors to help identify targets. In some examples, that may include a human interacting with a graphical user interface “GUI” such as a touchscreen to identify a target displayed on the screen
In any of the above examples of automation, the sensors onboard the harvesting system may be used to create, track and pass coordinates of the targets for harvesting.
Example Computer Device(s)
In example systems described herein, various computer components may be utilized to operate the systems. For example, a communication computer system may allow for remote operation of the machines, sensors may send information to a computer system to help differentiate targets from non-targets, target location and mapping information may be calculated, stored, sent, and utilized between the seeker systems and harvesting systems, steering and driving instructions may be calculated and utilized, machine learning/artificial intelligence/and/or neural networks may be employed by computer systems to find and harvest targets, and any of the other computer operations as described herein.
In some examples, alternatively or additionally, a WiFi system/cellular system/Bluetooth system, or any other communication system, with the appropriate antenna system and a processor and memory as described herein, may be used on a subassembly. In some embodiments, alternatively or additionally, the hardware may include a single integrated circuit containing a processor core, memory, and programmable input/output peripherals.
In some examples, various computer components may be used in the seeker and/or harvesting subassemblies, as well as the communication systems, control systems, and/or any other portion of the systems described herein.
FIG. 15 shows an example computer device 1500 that may be used in practicing example embodiments described herein. Such computer device 1500 may be the back end server systems use to interface with the network, receive and analyzed data, as well as generate test result GUIs. Such computer 1500 may be a mobile device used to create and send data, as well as receive and cause display of GUIs representing data. In FIG. 15, the computer device could be a smartphone, a laptop, tablet computer, server computer, or any other kind of computer device. The example shows a processor CPU 1510 which could be any number of processors in communication via a bus 1512 or other communication with a user interface 1514. The user interface 1514 could include any number of display devices 1518 such as a screen. The user interface also includes an input such as a touchscreen, keyboard, mouse, pointer, buttons, joystick or other input devices. Also included is a network interface 1520 which may be used to interface with any wireless or wired network in order to transmit and receive data. Such an interface may allow for a smartphone, for example, to interface a cellular network and/or WiFi network and thereby the Internet. The example computer device 1500 also shows peripherals 1524 which could include any number of other additional features such as but not limited to cameras, sensors 1525, and/or antennae 1526 for communicating wirelessly such as over cellular, WiFi, NFC, Bluetooth, infrared, or any combination of these or other wireless communications. The computer device 1500 also includes a memory 1522 which includes any number of operations executable by the processor 1510. The memory in FIG. 15 shows an operating system 1532, network communication module 1534, instructions for other tasks 1538 and applications 1538 such as send/receive message data 1540 and/or SMS text message applications 1542. Also included in the example is for data storage 1558. Such data storage may include data tables 1560, transaction logs 1562, user data 1564 and/or encryption data 1570. The computer device 1500 also include one or more graphical processing units (GPUs) for the purposes of accelerating in hardware computationally intensive tasks such as execution and or evaluation of the neural network engine and enhanced image exploitation algorithms operating on the multi-modal imagery collected. The computer device 1500 may also include one or more reconfigurable hardware elements such as a field programmable gate array (FPGA) for the purposes of hardware acceleration of computationally intensive tasks.
The computer architecture for the harvester can be described as a distributed computer system comprising of elements or processing centers that exist on the harvester, a central server system which may or may not be a cloud based resource and an operator processing system. Each of these processing centers are interconnected through an IP network which may include local private wireless networks, private wide area networks and/or public networks such as the Internet. Computational tasks are divided such that real-time tasks are executed on the local harvester processor, post-processing operations and non-real time computation are executed on the central server and user-interface computation are performed on the operator processing center.

CONCLUSION

As disclosed herein, features consistent with the present inventions may be implemented by computer-hardware, software and/or firmware. For example, the systems and methods disclosed herein may be embodied in various forms including, for example, a data processor, such as a computer that also includes a database, digital electronic circuitry, firmware, software, computer networks, servers, or in combinations of them. Further, while some of the disclosed implementations describe specific hardware components, systems and methods consistent with the innovations herein may be implemented with any combination of hardware, software and/or firmware. Moreover, the above-noted features and other aspects and principles of the innovations herein may be implemented in various environments. Such environments and related applications may be specially constructed for performing the various routines, processes and/or operations according to the invention or they may include a computer or computing platform selectively activated or reconfigured by code to provide the necessary functionality. The processes disclosed herein are not inherently related to any particular computer, network, architecture, environment, or other apparatus, and may be implemented by a suitable combination of hardware, software, and/or firmware. For example, various machines may be used with programs written in accordance with teachings of the invention, or it may be more convenient to construct a specialized apparatus or system to perform the required methods and techniques.
Aspects of the method and system described herein, such as the logic, may be implemented as functionality programmed into any of a variety of circuitry, including programmable logic devices (“PLDs”), such as field programmable gate arrays (“FPGAs”), programmable array logic (“PAL”) devices, electrically programmable logic and memory devices and standard cell-based devices, as well as application specific integrated circuits. Some other possibilities for implementing aspects include: memory devices, microcontrollers with memory (such as 1PROM), embedded microprocessors, Graphics Processing Units (GPUs), firmware, software, etc. Furthermore, aspects may be embodied in microprocessors having software-based circuit emulation, discrete logic (sequential and combinatorial), custom devices, fuzzy (neural) logic, quantum devices, and hybrids of any of the above device types. The underlying device technologies may be provided in a variety of component types, e.g., metal-oxide semiconductor field-effect transistor (“MOSFET”) technologies like complementary metal-oxide semiconductor (“CMOS”), bipolar technologies like emitter-coupled logic (“ECL”), polymer technologies (e.g., silicon-conjugated polymer and metal-conjugated polymer-metal structures), mixed analog and digital, and so on.
It should also be noted that the various logic and/or functions disclosed herein may be enabled using any number of combinations of hardware, firmware, and/or as data and/or instructions embodied in various machine-readable or computer-readable media, in terms of their behavioral, register transfer, logic component, and/or other characteristics. Computer-readable media in which such formatted data and/or instructions may be embodied include, but are not limited to, non-volatile storage media in various forms (e.g., optical, magnetic or semiconductor storage media) and carrier waves that may be used to transfer such formatted data and/or instructions through wireless, optical, or wired signaling media or any combination thereof. Examples of transfers of such formatted data and/or instructions by carrier waves include, but are not limited to, transfers (uploads, downloads, e-mail, etc.) over the Internet and/or other computer networks by one or more data transfer protocols (e.g., HTTP, FTP, SMTP, and so on).
Unless the context clearly requires otherwise, throughout the description and the claims, the words “comprise,” “comprising,” and the like are to be construed in an inclusive sense as opposed to an exclusive or exhaustive sense; that is to say, in a sense of “including, but not limited to.” Words using the singular or plural number also include the plural or singular number respectively. Additionally, the words “herein,” “hereunder,” “above,” “below,” and words of similar import refer to this application as a whole and not to any particular portions of this application. When the word “or” is used in reference to a list of two or more items, that word covers all of the following interpretations of the word: any of the items in the list, all of the items in the list and any combination of the items in the list.
Although certain presently preferred implementations of the invention have been specifically described herein, it will be apparent to those skilled in the art to which the invention pertains that variations and modifications of the various implementations shown and described herein may be made without departing from the spirit and scope of the invention. Accordingly, it is intended that the invention be limited only to the extent required by the applicable rules of law.
The foregoing description, for purpose of explanation, has been described with reference to specific embodiments. However, the illustrative discussions above are not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations are possible in view of the above teachings. The embodiments were chosen and described in order to best explain the principles of the invention and its practical applications, to thereby enable others skilled in the art to best utilize the invention and various embodiments with various modifications as are suited to the particular use contemplated.
The foregoing description, for purpose of explanation, has been described with reference to specific embodiments. However, the illustrative discussions above are not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations are possible in view of the above teachings. The embodiments were chosen and described in order to best explain the principles of the invention and its practical applications, to thereby enable others skilled in the art to best utilize the invention and various embodiments with various modifications as are suited to the particular use contemplated. Etc.

Claims

What is claimed is:

1. A system for harvesting agriculture, comprising:

a traversing machine with a frame, an articulating robotic arm attached to the frame, and a computer system attached to the frame with a processor and memory, the computer system in communication with the articulating robotic arm;

wherein the articulating robotic arm including at least two joints and at least one picker subassembly, the picker subassembly including at least one actuator in communication with the computer system;

wherein the picker subassembly includes a vacuum subassembly, the vacuum subassembly in communication with the computer system, the vacuum subassembly coupled to a nozzle with a terminal end, wherein the terminal nozzle end includes a flexible baffle section;

wherein the picker subassembly further includes two grappler spoons, the grappler spoons configured to pinch together toward the vacuum nozzle to secure a target by the actuator, in response to commands from the computer system.

2. The system of claim 1 wherein the picker subassembly vacuum nozzle is mounted on an extender actuator, in communication with the computer system, the extender actuator configured to extend away from the picker subassembly and back toward the picker subassembly, the grappler spoons configured to pinch toward the nozzle terminal end when the vacuum nozzle is retracted.

3. The system of claim 1 wherein the flexible baffle section is removable and friction fit to the picker subassembly and made of silicone.

4. The system of claim 1 wherein the grappler spoons are each attached to the picker subassembly by a flange and include at least one rim;

wherein the grappler spoons each include a resilient membrane stretched over the at least one rim.

5. The system of claim 4 wherein the grappler spoon membrane is removable and friction fit to the rim and made of silicone.

6. The system of claim 1 wherein the vacuum subassembly nozzle terminal end is generally round in cross section and includes a resilient end membrane with apertures to allow air to flow.

7. The system of claim 6 wherein the vacuum assembly nozzle terminal end apertures include at least four radially extending slots configured in a middle of the round resilient membrane.

8. The system of claim 1 wherein the picker subassembly is configured to twist in relation to the robotic arm, such that the twist might break a target stem when grappled by the terminal nozzle end and spoons.

9. The system of claim 1 wherein the picker subassembly includes a stem cutting saw configured around the picker subassembly, the stem cutting saw configured to slide around the picker subassembly and spin, relative to the picker subassembly.

10. The system of claim 1 further comprising at least two stereo cameras mounted to the frame, in communication with the computer system, configured to send image data regarding potential agricultural targets to the computer system for analysis.

11. A system for harvesting agriculture, comprising:

a traversing machine with a frame;

at least two wheels or tracks mounted to the frame;

a computer system with a processor and memory mounted to the frame;

a sensor mounted to the frame in communication with the computer system;

an articulating robotic arm in communication with the computer system, the articulating robotic arm mounted to the frame at a first end;

a pair of pincher spoons mounted to a second end of the articulating robotic arm, the pair of pincher spoons in communication with a pinching actuator, the pinching actuator in communication with the computer system and configured to pinch the pair of pincher spoons toward one another;

a vacuum system including a pump and a hose, the pump in communication with the computer system; the hose including a terminal end mounted at the second end of the articulating robotic arm, between the pair of pincher spoons.

12. The system of claim 1 wherein the terminal end of the vacuum hose includes a pliable baffle configured to flex in use.

13. The system of claim 12 wherein the pliable baffle is removably attached at a first end to the terminal end of the vacuum hose.

14. The system of claim 12 wherein the terminal end of the vacuum hose includes an actuator and extendible section configured to extend and retract between the pair of pincher spoons, the actuator in communication with the computer system.

15. The system of claim 13 wherein the pliable baffle includes a second end with slots configured to allow air to pass.

16. The system of claim 15 wherein the slots in the pliable baffle second end are radially configured or concentrically configured.

17. The system of claim 11 wherein the pincher spoons each include an inner and outer rim of pliable material.

18. The system of claim 1 wherein the pair of pincher spoons include a removable pliable membrane.

19. The system of claim 14 wherein the limit of retraction of the terminal end of the vacuum hose is past and above the pair of pincher spoons.

20. A method of harvesting agriculture, comprising:

traversing a harvester machine down a planter bed row of agriculture;

receiving, at a computer system on the harvester machine, image data of the agriculture;

determining, at the computer system, likely targets for harvesting from the image data;

sending coordinates of the likely targets to a robotic arm attached at a first end to the harvester machine;

sending harvest commands to a picker assembly mounted at a second end of the robotic arm, the harvest commands including activating a vacuum pump to draw air through a vacuum tube with a vacuum terminal end on the picker assembly, and actuating pinching of two pincher spoons mounted around the vacuum terminal end.

21. The method of claim 20 wherein the vacuum terminal end includes a flexible baffle section with apertures to allow air to pass through the vacuum tube.