TWI909003B - AN APPARATUS, A METHOD AND A COMPUTER PROGRAM PRODUCT FOR HANDLING A MIXTURE OF MATERIALS COMPRISING A PLURALITY OF DIFFERENT CLASSES OF MATERIALs - Google Patents

Abstract

A material sorting system sorts materials utilizing multiple stages of classification and sorting, including a vision system that implements a machine learning system in order to identify or classify each of the materials, and Laser Induced Breakdown Spectroscopy to perform a subsequent classification and sorting of the remaining materials.

Description

An apparatus, method, and computer program product for processing mixtures of materials comprising multiple different material categories

This application is a partial continuation of U.S. Patent Application No. 17/380,928, which is a partial continuation of U.S. Patent Application No. 17/227,245, which is a partial continuation of U.S. Patent Application No. 16/939,011, which is a partial continuation of U.S. Patent Application No. 16/375,675 (granted as U.S. Patent No. 10,722,922), and is a partial continuation of U.S. Patent Application No. 15/963,75. A partial continuation of U.S. Patent No. 10,710,119, which claims priority to U.S. Provisional Patent Application No. 62/490,219, and which is a partial continuation of U.S. Patent Application No. 15/213,129 (granted to U.S. Patent No. 10,207,296), which claims priority to U.S. Provisional Patent Application No. 62/193,332, all of which are incorporated herein by reference.

This application is also a partial continuation of U.S. Patent Application No. 16/852,514, which is a divisional application of U.S. Patent Application No. 16/358,374 filed on March 19, 2019 (granted as U.S. Patent No. 10,625,304), both of which are incorporated herein by reference. [Government License]

This disclosure was made with the support of the U.S. government and under award number DE-AR0000422 from the U.S. Department of Energy. The U.S. government may hold certain rights in this disclosure.

This disclosure relates generally to the sorting of materials, and more specifically to the sorting of materials using multi-stage sorting.

This section aims to introduce various forms of the art, which may be associated with exemplary embodiments of this disclosure. This discussion is considered helpful in providing a framework conducive to a better understanding of specific aspects of this disclosure. Therefore, this section should be understood to be read from this perspective, rather than necessarily as an endorsement of prior art.

Recycling is the process of collecting and processing materials that would otherwise be discarded as waste and transforming them into new products. Recycling benefits communities and the environment by reducing the amount of waste sent to landfills and incinerators, protecting natural resources, improving economic security by utilizing domestic material sources, preventing pollution by reducing the need to collect new raw materials, and saving energy. After collection, recyclable materials are typically sent to material recycling facilities for sorting, cleaning, and processing into materials that can be used in manufacturing.

Recycling aluminum (Al) waste is a highly attractive proposition because it can save up to 95% in energy costs associated with manufacturing compared to the laborious and costly extraction of primary aluminum. Primary aluminum is defined as aluminum derived from aluminum-rich ores such as bauxite. Meanwhile, demand for aluminum in markets such as automotive manufacturing is steadily increasing due to its lightweight properties. Therefore, the aluminum industry can reap economic benefits by developing a well-planned yet simple recycling program or system. The use of recycled materials will be a cheaper metal resource compared to primary aluminum sources. As the amount of aluminum sold to the automotive industry (and other industries) increases, the availability of using recycled aluminum to supplement primary aluminum will become increasingly necessary.

Correspondingly, there is a particular desire to effectively separate scrap aluminum into alloy families, as mixed aluminum scrap of the same alloy family is far more valuable than indiscriminately mixed alloys. For example, in mixing methods used for aluminum recycling, any quantity of scrap composed of similar or identical alloys and of consistent quality is more valuable than scrap composed of mixed aluminum alloys. In these aluminum alloys, aluminum is always the main component of the material. However, the inclusion of copper, magnesium, silicon, iron, chromium, zinc, manganese, and other alloying elements provides a range of properties to aluminum alloys and offers a method for distinguishing one aluminum alloy from another.

The Aluminium Industry Association (AIA) is the authoritative body defining permissible limits for the chemical composition of aluminum alloys. Information on the chemical composition of forging aluminum alloys is published by the AIA in “International Alloy Nomenclature and Chemical Composition Limits for Forged Aluminum and Forged Aluminum Alloys,” updated in January 2015, and incorporated herein by reference. Generally speaking, according to the Aluminum Industry Association, 1xxx series forged aluminum alloys are mainly composed of pure aluminum, with an aluminum content of at least 99% by weight; 2xxx series are forged aluminum alloys mainly alloyed with copper (Cu); 3xxx series are forged aluminum alloys mainly alloyed with manganese (Mn); 4xxx series are forged aluminum alloys made of silicon (Si); 5xxx series are forged aluminum alloys mainly alloyed with magnesium (Mg); 6xxx series are forged aluminum alloys mainly alloyed with magnesium and silicon; 7xxx series are forged aluminum alloys mainly alloyed with zinc (Zn); and 8xxx series are miscellaneous.

The Aluminium Industry Association also has similar documents regarding cast aluminum alloys. The 1xx series cast aluminum alloys are primarily composed of pure aluminum, with an aluminum content of at least 99% by weight; the 2xx series are cast aluminum alloys primarily with copper; the 3xx series are cast aluminum alloys primarily with silicon-copper and/or magnesium; the 4xx series are cast aluminum alloys primarily with silicon; the 5xx series are cast aluminum alloys primarily with magnesium; the 6xx series is an unused series; the 7xx series are cast aluminum alloys primarily with zinc; the 8xx series are cast aluminum alloys primarily containing tin; and the 9xx series are cast aluminum alloys with other elements. Examples of casting alloys used for automotive parts include 380, 384, 356, 360, and 319. For example, recycled casting alloys 380 and 384 can be used to manufacture vehicle engine blocks, transmissions, etc. Recycled casting alloy 356 can be used to manufacture aluminum alloy wheel hubs. Furthermore, recycled casting alloy 319 can be used to manufacture drive blocks.

Generally speaking, forged aluminum alloys have a higher magnesium content than cast aluminum alloys, and cast aluminum alloys have a higher silicon content than forged aluminum alloys.

Furthermore, the presence of mixed lumps of different alloys in the waste limits its ability to be effectively recycled unless the different alloys (or at least alloys belonging to different compositional groups, such as those specified by the Aluminium Industry Association) can be separated before remelting. This is because when mixed waste containing multiple different alloy compositions or compositional groups is remelted, the resulting molten mixture contains a proportion of major alloys and elements (or different components) that is too high to meet the compositional limits required in any particular commercial alloy. Additionally, as demonstrated by the production and sales of the Ford F-150 pickup truck, its body and frame components are made of aluminum instead of steel, thus necessitating the recycling of sheet metal waste (e.g., forged aluminum components of certain alloys), including components generated during the manufacture of automotive parts from aluminum sheets. Scrap recycling involves remelting scrap to provide molten metal, which can be cast and/or rolled into useful aluminum parts for further production of such vehicles. However, automotive manufacturing scrap (as well as metal scrap from other sources such as aircraft and commercial and household appliances) typically includes scrap parts of forged and cast parts and/or mixtures of two or more aluminum alloys that are significantly different from each other in composition. Therefore, those with ordinary knowledge in the field of aluminum alloys will understand the difficulty of separating aluminum alloys, especially processed alloys such as those cast, forged, extruded, rolled, and commonly forged alloys, into reusable or recyclable processed products.

In view of the purpose of this invention, the present invention provides an apparatus for processing a first mixture of materials comprising multiple different material categories, the apparatus comprising: an image sensor configured to capture visual observation features of each of the materials in the first mixture; and a data processing system including a machine learning system implementing a neural network configured with a previously generated set of neural network parameters, classifying a first plurality of materials in the first mixture into a first category of materials based on the captured visual observation features, wherein the previously generated set of neural network parameters is uniquely associated with the first category of materials, wherein the plurality of materials in the first mixture are classified into the first category of materials, having a chemical composition different from that of the materials in the first mixture, and not classified into the first category of materials.

This document discloses various detailed embodiments of the present disclosure. However, it should be understood that the disclosed embodiments are merely examples of the present disclosure and may be embodied in various and alternative forms. These figures are not necessarily drawn to scale; certain features may be exaggerated or minimized to show details of a particular component. Therefore, the specific structural and functional details disclosed herein should not be construed as limiting, but only as a representative basis for teaching those of ordinary skill in the art to adopt the various embodiments of the present disclosure.

As used herein, “chemical element” means a chemical element in the periodic table of elements, including chemical elements that may be discovered after the filing date of this application. As used herein, “material” can include a solid composed of a compound or mixture of one or more chemical elements, or a solid composed of a compound or mixture of compounds or mixtures of chemical elements, wherein the complexity of the compounds or mixtures can range from simple to complex (all of these may also be referred to herein as materials having a particular “chemical composition”). Material categories may include metals (ferrous and nonferrous), metal alloys, plastics (including but not limited to PCBs, HDPE, UHMWPE, and various colored plastics), rubber, foams, and glass (including but not limited to borosilicate or sodium-calcium glass). Glass (including various colored glass), ceramics, paper, cardboard, Teflon, PE, bundled wires, insulated wires, rare earth elements, leaves, wood, plants, plant parts, textiles, biowaste, packaging, electronic waste, batteries and accumulators, scrapped vehicles, mining, construction and demolition waste, agricultural waste, forest residues, specialty grasses, woody energy crops, microalgae, urban food waste, food waste, hazardous chemicals Chemical and biomedical waste, construction waste, agricultural waste, biological articles, non-biological articles, carbonaceous articles, any other articles that may be found in municipal solid waste, and any other articles, objects, or materials disclosed herein, including any other type or class that can be distinguished by one or more sensors, including but not limited to any sensing technology disclosed herein, including but not limited to one or more sensors. As used herein, the term “aluminum” means aluminum metals and aluminum-based alloys, i.e., alloys containing more than 50% by weight of aluminum (including those classified by the Aluminium Industry Association). In this disclosure, the terms “waste,” “waste part,” “material,” “material part,” and “part” are used interchangeably. As used herein, a material part or scrap part referred to as having a metal alloy composition is a metal alloy having a specific chemical composition that distinguishes it from other metal alloys.

According to the definition in the Nonferrous Scrap Guidelines published by the American Society of Recycling Industries (ISRI), the term "Zorba" is a collective term for shredded nonferrous metals, including but not limited to metal scrap ("ELV") from end-of-life vehicles or waste electronic and electrical devices ("WEEE"). ISRI has established the Zorba specification. In a Zorba, each piece of scrap may consist of a combination of nonferrous metals: aluminum, copper, lead, magnesium, stainless steel, nickel, tin, and zinc, in elemental or alloy (solid) form. Additionally, the term "Twitch" refers to shredded aluminum scrap. Twitch can be produced using a float process, in which aluminum scrap floats to the top while heavier metal scraps sink (for example, in some processes, sand may be mixed in to change the density of the water submerging the scrap).

As used in this article, the terms “identification” and “classification”, as well as their derivatives, are used interchangeably. As used in this article, “classification” refers to determining the type or category of material to which a piece of material belongs. For example, according to certain embodiments of this disclosure, a vision system or sensing system (as further described herein) may be configured to collect any type of information for classifying materials, which may be used within a sorting system to selectively sort material pieces as a function of one or more physical and/or chemical properties (e.g., user-defined), including but not limited to color, texture, hue, shape, brightness, weight, density, chemical composition, size, uniformity, type of manufacture, chemical characteristics, radioactivity, transmittance to light, sound or other signals, and response to stimuli such as various fields, including the emission and/or reflection of electromagnetic radiation (“EM”) by the material pieces.

The type or category of a material (i.e., classification) can be user-definable and is not limited to any known material classification. The granularity of a type or category can range from very coarse to very fine. For example, a type or category can include plastics, ceramics, glass, metals, and other materials with relatively coarse granularity; different metals and metal alloys, such as zinc, copper, brass, chromium, and aluminum, where the granularity is finer; or between specific types of plastics, where the granularity is relatively fine. Thus, a type or category can be configured to distinguish materials with significantly different chemical compositions, such as plastics and metal alloys, or to distinguish materials with nearly identical chemical compositions, such as different types of metal alloys. It should be understood that the methods and systems discussed in this paper can be used to accurately identify/classify materials whose chemical composition is completely unknown prior to classification.

As used in this article, “manufacturing type” refers to the type of manufacturing process of the material in a manufacturing component, such as metal parts formed by forging, casting (including but not limited to one-time mold casting, permanent mold casting and powder metallurgy), forging, material removal processes, extrusion, etc.

As mentioned herein, a "conveyor system" can be any known mechanical handling device that moves material from one location to another, including but not limited to aircraft conveyors, automotive conveyors, belt conveyors, belt-driven power roller conveyors, bucket conveyors, chain conveyors, chain-driven power roller conveyors, traction conveyors, and dust conveyors. Electric rail vehicle systems, flexible conveyors, gravity conveyors, gravity sliding conveyors, spool roller conveyors, electric roller conveyors, overhead I-beam conveyors, land conveyors, pharmaceutical conveyors, plastic belt conveyors, pneumatic conveyors, spiral or helical conveyors, helical conveyors, pipe gallery conveyors, vertical conveyors, vibrating conveyors, and metal wire mesh conveyors.

The material sorting system described herein according to certain embodiments of this disclosure receives a heterogeneous mixture of multiple material parts, wherein at least one material in the heterogeneous mixture comprises a composition different from one element (e.g., a metal alloy composition) or more other materials. Although all embodiments of this disclosure can be used to sort any type or class of materials as defined herein, certain embodiments of this disclosure are described below for sorting metal alloy material parts, including aluminum alloy material parts, and including forged, extruded, and/or cast aluminum alloy material parts.

It should be noted that the materials to be sorted may have irregular sizes and shapes (e.g., see Figures 6-8). For example, such materials (e.g., Zorba and/or Twitch) may have previously been processed by a shredding mechanism that cuts the material into fragments of these irregular shapes and sizes (producing waste pieces) before being fed or transferred to a conveyor system.

Embodiments of this disclosure will be described herein as sorting material parts into separate groups or sets by physically depositing (e.g., ejecting or transferring) them into individual containers or boxes or onto another conveying system, as a function of user-defined grouping or settling (e.g., material type classification). As an example, in some embodiments of this disclosure, material parts may be sorted to separate material parts composed of one or more specific chemical components from other material parts composed of different specific chemical components.

Furthermore, certain embodiments of this disclosure can sort aluminum alloy parts into individual bins, such that substantially all aluminum alloy parts having a chemical composition falling into one of the aluminum alloy series published by the Aluminium Industry Association are sorted into individual bins (e.g., a bin may correspond to one or more specific aluminum alloy series (e.g., 1000, 2000, 3000, 4000, 5000, 6000, 7000, 8000, 100, 200, 300, 400, 500, 600, 700, 800, 900)). Additionally, as will be described herein, certain embodiments of this disclosure can be configured to sort aluminum alloy parts into individual bins according to their alloy composition classification, even if such alloy compositions belong to the same Aluminium Industry Association series. As a result, the sorting system according to certain embodiments of this disclosure can classify and sort aluminum alloy parts having a composition that classifies them all into a single aluminum alloy series (e.g., series 300 or series 500) into separate bins as a function of their aluminum alloy composition. In a non-limiting example, certain embodiments of this disclosure can classify and sort aluminum alloy parts into separate bins, with aluminum alloy parts classified as cast aluminum alloy 319 and aluminum alloy parts classified as cast aluminum alloy 380, respectively.

Figure 1 illustrates an example of a system 100 configured to automatically sort/collect materials according to various embodiments of the present disclosure. A conveyor system 103 may be implemented to convey individual material pieces 101 through the system 100, enabling tracking, sorting, and/or collection of each individual material piece 101 within a predetermined desired group or set. Such a conveyor system 103 may be implemented using one or more conveyor belts, on which the material pieces 101 typically travel at a predetermined constant speed. However, certain embodiments of the present disclosure may be implemented using other types of conveyor systems disclosed herein. Hereinafter, the conveyor system 103 may also be referred to as a conveyor belt 103, as applicable. In one or more embodiments, some or all of the actions of transmission, stimulation, detection, sorting, and sorting can be performed automatically, i.e., without human intervention. For example, in system 100, one or more stimulation sources, one or more emission detectors, a sorting module, a sorting device, and/or other system components can be configured to perform these and other operations automatically.

Furthermore, although Figure 1 depicts a single material stream 101 on conveyor belt 103, embodiments of this disclosure can be implemented in which multiple such material streams pass parallel to each other through the components of system 100, or a collection of material items randomly deposited onto the conveyor system (e.g., conveyor belt 103) passes through the components of system 100. Therefore, certain embodiments of this disclosure can simultaneously track, classify, and/or sort multiple such parallel material streams, or material items randomly deposited on the conveyor system (belt). However, according to embodiments of this disclosure, the segmentation of material item 101 does not require tracking, classifying, and/or sorting of material items.

The conveyor belt 103 may be a conventional endless belt conveyor, employing a conventional drive motor 104 suitable for moving the conveyor belt 103 at a predetermined speed. According to certain embodiments of this disclosure, a suitable feeder mechanism may be used to feed material 101 onto the conveyor belt 103, thereby conveying the material 101 through various components within the system 100. In certain embodiments of this disclosure, the conveyor belt 103 is operated by the conveyor belt motor 104 to travel at a predetermined speed. This predetermined speed can be programmed and/or adjusted by an operator in any well-known manner. In certain embodiments of this disclosure, control of the conveyor belt motor 104 and/or position detector 105 may be performed by an automation control system 108. Such an automation control system 108 can operate under the control of a computer system 107 and/or the functions used to perform automation control can be implemented in the software within the computer system 107.

The position detector 105, which may be a conventional encoder, can be operatively coupled to the conveyor belt 103 and the automation control system 108 to provide information corresponding to the motion (e.g., speed) of the conveyor belt 103. Thus, as will be further described herein, by using control of the conveyor belt drive motor 104 and/or the automation control system 108 (and optionally including the position detector 105), when each material piece 101 is identified traveling on the conveyor belt 103, they can be tracked by position and time (relative to the components of system 100) so that the components of system 100 can be triggered/deactivated when each material piece 101 passes in their vicinity. As a result, the automation control system 108 is able to track the position of each material piece 101 as it travels along the conveyor belt 103.

According to certain embodiments of this disclosure, after material items 101 are received by conveyor belt 103, individual material items can be separated from the collection of material items using rollers and/or vibrators, and then they can be positioned into one or more individual (i.e., individual file) streams. According to alternative embodiments of this disclosure, material items can be positioned into one or more individual (i.e., individual file) streams, which can be performed by an active or passive sorter 106. An example of a passive sorter is further described in U.S. Patent No. 10,207,296. As previously stated, there is no need to incorporate or use a sorter. Instead, a conveyor system (e.g., conveyor belt 103) can simply convey a collection of material items already randomly placed on conveyor belt 103.

Referring again to Figure 1, certain embodiments of this disclosure may utilize a visual or optical recognition system 110 and/or a distance measuring device 111 as a means of initiating tracking of each material piece 101 traveling on the conveyor belt 103. The visual system 110 may utilize one or more stationary or live cameras 109 to record the position (i.e., location and time) of each material piece 101 on the moving conveyor belt 103. The visual system 110 may be further or alternatively configured to perform certain types of identification (e.g., classification) on all or some of the material pieces 101. For example, such a visual system 110 may be used to acquire information about each material piece 101. For example, the vision system 110 can be configured (e.g., having a machine learning system) to collect any type of information that can be used within the system 100 to classify material part 101 as a set of one or more (user-defined) functions of physical properties, including, but not limited to, the color, hue, size, shape, texture, overall physical appearance, uniformity, composition, and/or manufacturing type of material part 101. The vision system 110 captures images (including one-dimensional, two-dimensional, three-dimensional, or holographic images) of each material part 101, for example, using optical sensors typically used in digital cameras and video devices. These images captured by the optical sensors are then stored as image data in a memory device. According to embodiments of this disclosure, such image data represents an image captured within the optical wavelength of light (i.e., the wavelength of light that is typically observable by the human eye). However, alternative embodiments of this disclosure may utilize a sensor capable of capturing images of materials composed of light wavelengths outside the visual wavelengths of the typical human eye.

According to certain embodiments of this disclosure, one or more sensor systems 120 may be used alone or in conjunction with vision system 110 to classify/identify material parts 101. Sensor system 120 may be configured with any type of sensor technology, including sensors utilizing radiated or reflected electromagnetic radiation (e.g., infrared (“IR”), Fourier transform IR (“FTIR”), forward-looking infrared (“FLIR”), very near-infrared (“VNIR”), near-infrared (“NIR”), short-wave infrared (“SWIR”), long-wave infrared (“LWIR”), mid-wave infrared (“MWIR”), X-ray transmission (“XR”). XRF (“XRF”), gamma rays, ultraviolet rays, X-ray fluorescence (“XRF”), laser-induced breakdown spectroscopy (“LIBS”), Raman spectroscopy, anti-Stokes Raman spectroscopy, gamma spectroscopy, hyperspectral spectroscopy (e.g., any range beyond the visible wavelength), acousto-optic spectroscopy, NMR spectroscopy, microwave spectroscopy, terahertz spectroscopy, including one-dimensional, two-dimensional, or three-dimensional imaging having any of the foregoing), or by any other type of sensing technique, including but not limited to chemical or radioactive. An implementation of an XRF system (e.g., used herein as sensing system 120) is further described in U.S. Patent No. 10,207,296.

It should be noted that although Figure 1 is shown as a combination of vision system 110 and sensor system 120, embodiments of this disclosure can be implemented using any combination of sensor systems that utilize any of the sensor technologies disclosed herein, or any other sensor technologies currently available or to be developed in the future. Although Figure 1 is shown to include sensor system 120, such implementation of a sensor system is optional in some embodiments of this disclosure. In some embodiments of this disclosure, a combination of vision system 110 and one or more sensor systems 120 can be used to classify material part 101. In some embodiments of this disclosure, any combination of one or more different sensor technologies disclosed herein can be used to classify material part 101 without using vision system 110. Furthermore, embodiments of this disclosure may include any combination of one or more sensor systems and/or vision systems, wherein the output of such sensor and/or vision systems is utilized by a machine learning system (as further disclosed herein) to classify/identify materials from heterogeneous mixtures of materials, which can then be sorted together.

According to an alternative embodiment of this disclosure, the vision system 110 and/or the sensor system can be configured to identify which material items 101 are not of the type sorted by system 100 (sometimes referred to as contaminants) and send a signal to reject such materials. In such a configuration, the identified material items 101 can be transferred/discharged using one of the mechanisms described below for physically moving the sorted material items to individual bins.

In some embodiments of this disclosure, a distance measuring device 111 and an accompanying control system 112 may be utilized and configured to measure the size and/or shape of each of the material pieces 101 as they pass near the distance measuring device 111, along with the position (i.e., location and time) of each material piece 101 on the moving conveyor belt 103. Exemplary operation of such a distance measuring device 111 and control system 112 is further described in U.S. Patent No. 10,207,296. Alternatively, as previously disclosed, a vision system 110 may be used to track the position (i.e., location and time) of each material piece 101 on the moving conveyor belt 103.

This distance measuring device 111 can be implemented using a well-known visible light (e.g., laser) system that continuously measures the distance traveled by the light before it is reflected back to the detector of the laser system. Thus, as each piece of material 101 passes near the device 111, it outputs a signal to the control system 112 indicating this distance measurement. Therefore, such a signal can essentially represent an intermittent sequence of pulses, the baseline of which is generated as a result of the measured distance between the distance measuring device 111 and the conveyor belt 103 during the periods when the piece of material 101 is not near the device 111, with each pulse providing the measured distance between the distance measuring device 111 and the piece of material 101 passing on the conveyor belt 103. Because the material piece 101 may have an irregular shape, the height of such pulse signals may sometimes also be irregular. However, each pulse signal generated by the distance measuring device 111 provides a partial height of each material piece 101 as it passes over the conveyor belt 103. The length of each such pulse also provides a measurement of the length of each material piece 101, which is measured along a line substantially parallel to the direction of travel of the conveyor belt 103. In some embodiments of this disclosure, this length measurement (or height measurement) can be used to determine when the XRF system implemented by the sensor system 120 excites and deactivates the acquisition of detected fluorescence (i.e., XRF spectrum) for each material part 101, such that the detected fluorescence is obtained essentially only from each material part, and not from any background surface, such as conveyor belt 103. This results in more accurate fluorescence detection and analysis, and also saves signal processing time for the detected signal, since only data related to the fluorescence detected from the material parts must be processed.

In some embodiments of this disclosure implementing the sensor system 120, the sensor system 120 may be configured to assist the vision system 110 in identifying the chemical composition or relative chemical composition of each material element 101 as they pass in the vicinity of the sensor system 120. The sensor system 120 may include an energy emission source 121, for example, which may be powered by a power source 122, to excite a response from each material element 101.

In some embodiments of this disclosure, when each material element 101 passes near the emission source 121, the sensing system 120 can emit appropriate sensing signals to the material element 101. One or more detectors 124 can be positioned and configured to sense/detect one or more physical characteristics from the material element 101 in a form suitable for the type of sensing technology used. One or more detectors 124 and associated detector electronics 125 capture the received sensing characteristics to perform signal processing thereon and generate digital information representing the sensing characteristics, which is then analyzed according to some embodiments of this disclosure and can be used for classification of each material element 101 (alone or in combination with the vision system 110). This sorting, which can be performed within computer system 107, can then be utilized by automation control system 108 to trigger one of N (N≥1) sorting devices 126…129, according to a determined sorting method, for sorting (e.g., transferring/discharging) material pieces 101 to one or more N (N≥1) sorting bins 136…139. Four sorting devices 126…129 and four sorting bins 136…139 associated with the sorting devices are shown in Figure 1, as a non-limiting example only.

The sorting device may include any known mechanism for reorienting selected material items 101 to a desired location, including but not limited to transferring material items 101 from a conveyor system to multiple sorting bins. For example, the sorting device may utilize air ejectors, each assigned to one or more categories. When one of the air ejectors (e.g., 127) receives a signal from the automation control system 108, the air ejector emits a stream of air, causing material items 101 to be transferred/discharged from the conveyor system 103 to the sorting bin (e.g., 137) corresponding to that air ejection. For example, a high-speed air valve from Mac Industries can be used to provide appropriate air pressure for the air jet, which is configured to transfer/discharge the material part 101 from the conveying system 103.

Although the example shown in Figure 1 uses an air jet to transfer/discharge material items, other mechanisms can be used to transfer/discharge material items, such as a robot removing material items from a conveyor belt, pushing material items from a conveyor belt (e.g., with a paintbrush-type plunger), forming an opening (e.g., a trapdoor) in the conveyor system 103 from which the material items can fall, or using an air jet to separate the material items into individual containers as they fall from the edge of the conveyor belt. As used herein, an actuator device can refer to any form of device that can be triggered to dynamically move an object into or out of a conveying system/device, using pneumatic, mechanical, or other means, such as any suitable type of mechanical actuation mechanism (e.g., ACME screw drive), pneumatic actuation mechanism, or air jet actuation mechanism. Some embodiments may include multiple actuator devices located at different positions and/or with different turning path orientations along the path of the conveying system. In various embodiments, the sorting systems described herein can determine which actuator device (if any) to trigger based on the characteristics of the material object identified by the machine learning system. Furthermore, determining which actuator to trigger can be based on the presence and/or characteristics of other objects detected, which may also be within the transfer path of the actuator simultaneously with the target item. Additionally, even for facilities with imperfect sorting along the conveyor system, the disclosed sorting system can identify when multiple objects are not properly sorted and dynamically select from multiple actuators that should be triggered based on the actuator that provides the optimal transfer path for separating objects within close proximity. In some embodiments, the object identified as the target object may represent material that should be transferred out of the conveyor system. In other embodiments, the object identified as the target object represents material that should be allowed to remain on the conveyor system so that non-target material can be transferred.

In addition to the material item 101 being turned/ejected into the N sorting bins 136…139, the system 100 may also include a receiver or bin 140 that receives material items 101 that have not been turned/ejected from the transport system 103 into any of the aforementioned sorting bins 136…139. For example, material item 101 may not be turned/ejected from the transport system 103 into one of the N sorting bins 136…139 when the sorting of material item 101 is undetermined (or simply because the sorting device failed to turn/eject one item sufficiently), or when material item 101 contains contaminants detected by the vision system 110 and/or the sensor system 120. Therefore, bin 140 can be used as a pre-defined container for dumping unsorted material items into it. Alternatively, bin 140 can be used to receive material items that are intentionally not assigned to any of the N sorting bins 136…139, one or more of their categories. These material items can then be further sorted according to other characteristics and/or by another sorting system.

Depending on the diversity of the required material categories, multiple categories can be mapped to a single sorting device and associated sorting bins. In other words, a one-to-one correlation is not required between categories and sorting bins. For example, a user may want to sort certain categories of materials into the same sorting bins. To accomplish this sorting, when material item 101 is classified as falling into a predetermined category group, the same sorting device can be activated to sort them into the same sorting bin. This combined sorting can be used to produce any desired combination of sorted material items. The mapping of categories can be programmed by the user (e.g., using a sorting algorithm operated by computer system 107 (e.g., see Figure 5)) to produce such desired combinations. Furthermore, the classification of materials is user-defined and is not limited to any particular known material classification.

The conveying system 103 may include a circular conveyor (not shown) that returns unsorted material items to the starting point of the system 100 and passes through the system 100 again. Furthermore, because the system 100 is able to specifically track each material item 101 as it travels on the conveying system 103, a sorting device (e.g., sorting device 129) can be implemented to guide/expel material items 101 after a predetermined number of cycles through the system 100 where the system 100 fails to sort them (or material items 101 are collected in bin 140).

In some embodiments of this disclosure, the conveying system 103 may be divided into multiple belts configured in series, such as two belts, wherein the first belt conveys material items through the vision system 110 and/or the implemented sensor system 120, and the second belt conveys material items from the vision system 110 and/or the implemented sensor system 120 to a sorting device. Furthermore, the height of such a second conveyor belt may be lower than the height of the first conveyor belt, allowing material items to fall from the first conveyor belt onto the second conveyor belt.

In some embodiments of the present disclosure implementing the sensor system 120, the emission source 121 may be located above the detection area (i.e., above the transmission system 103); however, in some embodiments of the present disclosure, the emission source 121 and/or the detector 124 may be positioned at other locations that still produce acceptable sensing/detection physical characteristics.

As further described herein, using system 100 which implements an XRF system for sensor system 120, a signal representing the detected XRF spectrum can be converted into a discrete energy histogram such as that based on each channel (i.e., element). This conversion process can be implemented within control system 123 or computer system 107. In some embodiments of this disclosure, such control system 123 or computer system 107 may include a commercially available spectrum acquisition module, such as the commercially available Amptech MCA 5000 acquisition card and software programmed to operate that card. This spectral acquisition module, or other software implemented within system 100, can be configured to implement multiple channels for dispersing X-rays into a discrete energy spectrum (i.e., a histogram) with these multiple energy levels, whereby each energy level corresponds to an element that system 100 has been configured to detect. System 100 can be configured such that sufficient channels correspond to certain elements in the periodic table of chemistry, which is important for distinguishing different materials. The energy counts for each energy level can be stored in a separate collection register. Computer system 107 then reads each collection register to determine the counts for each energy level during collection intervals and constructs an energy histogram. As will be described in more detail herein, a sorting algorithm configured according to certain embodiments of this disclosure can then utilize the collected energy level histogram to classify at least some of the material items 101 and/or assist the visual system 110 in classifying the material items 101.

According to certain embodiments of this disclosure that implement an XRF system as a sensor system 120, source 121 may include a tandem X-ray fluorescence (“IL-XRF”) tube, as further described, for example, in U.S. Patent No. 10,207,296. Such an IL-XRF tube may include a single X-ray source, each dedicated to one or more (e.g., single-streamed) material pieces. In this case, one or more detectors 124 may be implemented as XRF detectors to detect fluorescent X-rays from material pieces 101 within each sorted stream. An example of such an XRF detector is further described in U.S. Patent No. 10,207,296.

It should be understood that although the systems and methods described herein are primarily for the classification of solid material components, this disclosure is not limited thereto. The systems and methods described herein can be used to classify materials having any range of physical states, including but not limited to liquids, melts, gases or powdered solids, other states, and any suitable combination thereof.

The systems and methods described herein can be applied to classifying and/or sorting individual pieces of material of any size, ranging from 1/4 inch or smaller in diameter. Although the systems and methods described herein are primarily described with respect to individual pieces of material being sorted one at a time in a stream, they are not limited thereto. Such systems and methods can be used to simultaneously stimulate and/or detect emission from multiple materials. For example, instead of a single material stream being conveyed in series along one or more conveyor belts, multiple sorted streams can be conveyed in parallel. Each stream can be on the same belt or on different belts arranged in parallel. Furthermore, fragments can be randomly distributed on one or more conveyor belts (e.g., across and along). Thus, the systems and methods described herein can be used to simultaneously stimulate and/or detect emission from multiple of these fragments. In other words, multiple small pieces can be considered as a single piece, rather than each piece being considered individually. Therefore, multiple small pieces of material can be sorted and grouped together (e.g., transferred/discharged from a conveyor system). It should be understood that multiple larger material pieces can also be considered as a single material piece.

As previously described, certain embodiments of this disclosure may implement one or more vision systems (e.g., vision system 110) for identifying, tracking, and/or sorting material items. According to embodiments of this disclosure, such a vision system may operate independently to identify and/or sort and pick material items, or it may operate in combination with a sensing system (e.g., sensing system 120) to identify and/or sort and pick material items. If a sorting system (e.g., system 100) is configured to operate only with such a vision system 110, then the sensing system 120 may be omitted from system 100 (or simply disabled).

This vision system can be configured with one or more devices for capturing or acquiring images of material parts as they pass over a transport system. The device can be configured to capture or acquire any desired wavelength range illuminated or reflected by the material parts, including but not limited to visible light, infrared (“IR”), and ultraviolet (“UV”) light. For example, the vision system can be configured with one or more cameras (stationary and/or video, any one of which can be configured to capture two-dimensional, three-dimensional, and/or holographic images) located near (e.g., above) the transport system to capture images of the material parts as they pass through the sensor system. According to an alternative embodiment of this disclosure, the data captured by the sensing system 120 can be processed (converted) into data for use (alone or in combination with image data captured by the vision system 110) in classifying/sorting materials. This embodiment can replace or be used in conjunction with the sensing system 120 to classify materials.

Regardless of the type of sensing features/information extracted from the material, the information can then be sent to a computer system (e.g., computer system 107) for processing by a machine learning system to identify and/or classify each material. Such a machine learning system can implement any well-known machine learning system, including machine learning systems that implement neural networks (e.g., artificial neural networks, deep neural networks, convolutional neural networks, recurrent neural networks, autoencoders, reinforcement learning, etc.), supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, self-learning, feature learning, sparse dictionary learning, anomaly detection, robot learning, association rule learning, fuzzy logic, and artificial intelligence. AI, deep learning algorithms, deep structured learning layered learning algorithms, support vector machines ("SVM") (e.g., linear SVM, nonlinear SVM, SVM regression, etc.), decision tree learning (e.g., classification and regression trees ("CART")), ensemble methods (e.g., ensemble learning, random forest, bagging and pasting, patching and subspaces, acceleration, stacking, etc.), dimensionality reduction (e.g., projection, manifold learning (Manifold)). Machine learning algorithms, such as those described and publicly available on the deeplearning.net website (including hyperlinks to all software, publications, and available software referenced hereinstantly), are incorporated herein by reference. Non-limiting examples of publicly available machine learning software and databases that may be used in embodiments of this disclosure include Python, OpenCV, Inception, Theano, Torch, PyTorch, Pylearn2, Numpy, Blocks, TensorFlow, MXNet, Caffe, Lasagne, Keras, Chainer, Matlab Deep Learning, CNTK, MatConvNet (a MATLAB toolbox implementing convolutional neural networks for computer vision applications), and DeepLearnToolbox (a Matlab toolbox for deep learning from Rasmus Berger). Palm), BigDL, Cuda-Convnet (a fast C++/CUDA implementation of convolutional (or more generally, feedforward) neural networks), Deep Belief Networks, RNNLM, RNNLIB-RNNLIB, matrbm, deeplearning4j, Eblearn.lsh, deepmat, MShadow, Matplotlib, SciPy, CXXNET, Nengo-Nengo, Eblearn, cudamat, Gnumpy, 3-way factored RBM and mcRBM, mPoT (Python code using CUDAmat and Gnumpy to train models of natural images), ConvNet, Elektronn, OpenNN, NeuralDesigner, Theano Generalized Hebbian Learning, Apache Singa, Lightnet, and SimpleDNN.

Machine learning typically occurs in two phases. For example, first, training occurs, which can be performed offline because system 100 is not used to perform the actual classification/sorting of material items. System 100 can be used to train a machine learning system where a homogeneous set of material items (i.e., materials of the same type or category) (also referred to herein as control samples) passes through system 100 (e.g., via transport system 103); and all these material items may not be sorted but can be collected in a common bin (e.g., bin 140). Alternatively, training can be performed at a location remote from system 100, including using some other mechanism to collect sensory information (features) of homogeneous groups of material items. In this training phase, algorithms in a machine learning system extract features from the captured information (e.g., using image processing techniques well-known in the field). Non-limiting examples of training algorithms include, but are not limited to, linear regression, gradient descent, feed forward, polynomial regression, learning curves, regularized learning models, and logical regression. It is during this training phase that the algorithms in the machine learning system learn the relationships between different types of materials and their characteristics/properties (e.g., captured by the vision system and/or sensor system), thereby establishing a knowledge base for later classification of heterogeneous mixtures of material items received by system 100, for sorting according to the desired classification. Such a knowledge base may include one or more databases, each of which includes parameters (also referred to herein as "neural network parameters") for the machine learning system to use when classifying material items. For example, a particular database may include parameters configured by the training phase to identify and classify specific types or categories of materials. According to certain embodiments of this disclosure, such a database can be input into a machine learning system, and then the user of system 100 can adjust certain parameters to adjust the operation of system 100 (e.g., adjusting the threshold validity of the machine learning system's ability to identify a particular material from a mixture of heterogeneous materials).

Furthermore, the presence of certain materials (e.g., chemical elements or compounds) or combinations of certain chemical elements or compounds in a material part (e.g., a metal alloy) results in identifiable physical characteristics (e.g., visually distinguishable properties) in the material. Consequently, when multiple material parts containing this specific composition pass through the aforementioned training phase, the machine learning system can learn how to distinguish these material parts from other material parts. Therefore, a machine learning system configured according to certain embodiments of this disclosure can be configured to sort among material parts as a function of their respective material/chemical composition. For example, such a machine learning system can be configured such that aluminum alloys can be sorted based on the percentage of a specific alloying material contained in them.

For example, Figure 2 shows an image of the extraction or acquisition of an exemplary cast aluminum material part that can be used during the aforementioned training phase. Figure 3 shows an image of the extraction or acquisition of an exemplary extruded aluminum material part that can be used during the aforementioned training phase. Figure 4 shows an image of the extraction or acquisition of an exemplary forged aluminum material part that can be used during the aforementioned training phase. During the training phase, multiple material parts of a specific (homogeneous) classification (type) of material, serving as control samples, can be transmitted through a transmission system to a vision system so that the machine learning system can detect, extract, and learn which features visually represent these typical material part segments. In other words, an image of a cast aluminum part, as shown in Figure 2, might first undergo a training phase so that the machine learning algorithm "learns" how to detect, identify, and classify parts composed of cast aluminum alloys. This will establish a parameter library specific to cast aluminum parts. Then, the same processing can be performed on images of extruded aluminum sheets, as shown in Figure 3, to establish a parameter library specific to extruded aluminum parts. Furthermore, the same processing can be performed on images of forged aluminum parts, as shown in Figure 4, to establish a parameter library specific to forged aluminum parts. For each type of material to be classified by the vision system, any number of exemplary material parts of that type can be processed by the vision system. Given the extracted image as input data, the machine learning algorithm can use N classifiers, each classifier testing one of N different material types.

After the algorithm is established and the machine learning system has fully understood the differences in material classification (e.g., within a user-defined statistical confidence level), a neural network parameter library for different materials is then implemented in a material classification and/or sorting system (e.g., system 100) to identify and/or classify material parts from heterogeneous mixtures of material parts, and then, if sorting is to be performed, these classified material parts may be sorted.

As seen in the relevant literature, the techniques for constructing, optimizing, and utilizing machine learning systems are known to those of ordinary knowledge in the field. Examples of such literature include the following publications: Krizhevsky et al., “ImageNet Classification with Deep Convolutional Networks,” Proceedings of the 25th International Conference on Neural Information Processing Systems, December 3-6, 2012, Lake Tahoe, Nevada; and LeCun et al., “Gradient-Based Learning Applied to Document Recognition,” IEEE Conference Proceedings, Institute of Electrical and Electronics Engineers (IEEE), November 1998, the entire contents of which are incorporated herein by reference.

In the example technique, data about a specific material object captured by a sensor and/or vision system can be processed as an array of data values. For example, the data could be image data captured by a digital camera or other type of imaging sensor about a specific material object and processed as an array of pixel values. Each data value can be represented by a single number or as a series of numerical values. These values are multiplied by neuron weight parameters, and possibly biased. This is fed into a neuronal nonlinearity. The resulting numerical output from the neurons can be processed like a value, multiplied by subsequent neuron weight values, optionally with bias added, and fed back into the neuronal nonlinearity. Each iteration of this process is called a “layer” of the neural network. The final output of the final layer can be interpreted as the probability of the material's presence or absence in the data related to the material. Examples of this process are described in detail in the previously mentioned references "ImageNet Classification with Deep Convolutional Networks" and "Gradient-Based Learning Applied to Document Recognition".

According to embodiments of this disclosure, as the final layer (“classification layer”), the final set of neural outputs is trained to represent the probability that a material item is associated with the extracted data. During operation, if the probability that a material item is associated with the extracted data exceeds a user-specified threshold, it is determined that a particular material item is indeed associated with the extracted data. These techniques can be extended to not only determine the existence of a material type associated with specific extracted data, but also to determine whether a subregion of specific extracted data belongs to one type of material or another type of material. This process is called segmentation, and techniques using neural networks exist in the literature, such as those called "fully convolutional" neural networks, or networks that include convolutional portions (i.e., partial convolutions), if not fully convolutional. This allows for the determination of the location and size of the material.

It should be understood that this disclosure is not exclusively limited to machine learning techniques. Other commonly used techniques for material classification/identification can also be used. For example, a sensor system can utilize spectral techniques using multispectral or hyperspectral cameras to provide signals that can indicate the presence or absence of a material by examining its spectral emission. Photographs of material parts can also be used in template-matching algorithms, where an image database is compared with the acquired images to determine the presence of certain types of materials from the database. Histograms of the extracted images can also be compared with a histogram database. Similarly, the bag-of-words model can be used with feature extraction techniques such as scale-invariant feature transform (“SIFT”) to compare extracted features between the extracted images and images in a database.

Therefore, as disclosed herein, certain embodiments of this disclosure provide for the identification/classification of one or more different materials in order to determine which material items should be transferred from a transport system or device. According to certain embodiments, machine learning techniques are used to train (i.e., configure) neural networks to identify one or more different materials. By capturing images or other types of sensory information about the material (e.g., traveling on the transport system), and based on the identification/classification of such materials, the system described herein can determine which material items should be allowed to remain in the transport system and should be transferred/removed from the transport system (e.g., into a collection bin, or transferred to another transport system).

According to certain embodiments of this disclosure, a machine learning system for an existing device can be dynamically reconfigured to detect and identify the properties of new materials by replacing the current set of neural network parameters with a new set of neural network parameters.

It should be noted that, according to certain embodiments of this disclosure, the features/properties of the material being detected/extracted are not necessarily simple, particularly identifiable physical properties; they may be abstract formulas that can only be expressed mathematically, or may not be expressed mathematically at all. Nevertheless, the machine learning system parses all the data to find patterns that allow for the classification of control samples during the training phase. Furthermore, the machine learning system can acquire sub-parts of the extracted information of the material and attempt to find correlations between predefined classifications.

According to certain embodiments of this disclosure, instead of a training phase using control samples of material parts through a vision system and/or sensor system, the training of the machine learning system can utilize labeling/annotation techniques (or any other supervised learning techniques) when the data/information of the material parts is captured by the vision/sensor system, the user inputs labels or annotations to identify each material part, and then classifies the material parts in a heterogeneous mixture to build a library for use by the machine learning system.

According to certain embodiments of this disclosure, any sensed feature output by any of the sensing systems 120 disclosed herein can be input into a machine learning system for classifying and/or sorting materials. For example, in a machine learning system implementing supervised learning, the output of the sensing system 120 can be used to train the machine learning system to uniquely characterize the composition of a particular type or material (e.g., a particular metal alloy).

After passing through a crusher, the wall panels (typically made of thin aluminum sheets), extrusions (typically made of thick aluminum frame strips), and castings look very different. Figure 2 shows a visual image of an exemplary scrap piece from cast aluminum. Figure 3 shows a visual image of an exemplary scrap piece from an aluminum extrusion. Figure 4 shows a visual image of an exemplary scrap piece from forged aluminum. Embodiments of this disclosure utilize a visual system, as described herein, capable of sorting/separating these three different types of aluminum scrap. As shown in the examples in Figures 2-4, aluminum extrusions have an overall physical appearance that can be distinguished from cast and forged aluminum scrap, which can be learned by a machine learning system configured according to embodiments of this disclosure.

This disclosure embodiment is configured to sort forged aluminum alloy parts from a Twitch that includes both forged and cast aluminum parts. In some embodiments of this disclosure, extruded aluminum alloy parts may be sorted together with forged aluminum alloy parts (or separately from cast and forged aluminum). Since most of the Mg is in forged aluminum, the remaining aluminum scrap (mainly consisting of cast aluminum alloys) has a relatively negligible amount of Mg. According to certain embodiments of this disclosure, these remaining aluminum scrap parts (also referred to herein as the "cast portion") can undergo further sorting (or multiple sorting cycles) to classify/sort and/or remove other impurities (e.g., scrap parts composed of PCBs, stainless steel, foam, rubber, etc.) among any number of different cast aluminum alloys. The cast portion may include casting alloys such as 319, 356, 360, and/or 380 series alloys. These alloys contain varying amounts of silicon, Cu, Zn, Fe, and Mn, but contain very small amounts of magnesium, typically 0-0.6%.

According to certain embodiments of this disclosure, one or more sensor systems 120 disclosed herein can be used to classify/sort one or both of the aforementioned casting and forging portions. For example, one or both of an XRF system and/or a LIBS-based sensor system can be used to classify/sort among two or more different cast aluminum alloys or two or more different forged aluminum alloys. This is done using an XRF system, as disclosed in U.S. Patent No. 10,207,296.

Spectroscopic techniques known as laser-induced breakdown spectroscopy (“LIBS”), laser spark spectroscopy (“LSS”), or laser-induced optical emission spectroscopy (“LIOES”) use a focused laser beam to evaporate and subsequently produce spectral lines emitting from the sample material. In this way, the chemical composition of a sample placed away from an analytical instrument can be analyzed. Embodiments of this disclosure can utilize any of the above to classify multiple materials into different categories for sorting. The use of LIBS for classification is further described in U.S. Patents 5,042,947, 6,545,240, and 10,478,861, all of which are incorporated herein by reference.

Figure 5 illustrates a flowchart of an exemplary embodiment of a process 3500 for classifying/sorting material items using a vision system and/or a sensor system according to certain embodiments of the present disclosure. Process 3500 can be configured to operate in any embodiment of the present disclosure described herein, including system 100 of Figure 1. Operation of process 3500 can be performed by hardware and/or software, including within a computer system (e.g., computer system 3400 of Figure 8) that controls a sorting system (e.g., computer system 107, vision system 110, and/or sensor system 120 of Figure 1). In process block 3501, material items are fed into a transport system. In process block 3502, the position of each material piece on the transport system is detected to track it as it travels through the sorting system. This can be performed by vision system 110 (e.g., by distinguishing the material pieces from the transport system materials below while communicating with a position detector in the transport system (e.g., position detector 105)). Alternatively, a linear sheet laser beam can be used to locate the fragments. Or, any system that can generate a light source (including but not limited to visible light, ultraviolet light, and IR) and has a detector that can be used to locate the fragments. In process block 3503, sensing information/characteristics of the material piece are captured/acquired as it has traveled close to one or more of the vision system and/or sensor systems. In process block 3504, a vision system (e.g., implemented within computer system 107), such as those previously disclosed, can perform preprocessing for extracting information, which can be used to detect (extract) each material element (e.g., from the background (e.g., the conveyor belt); in other words, the preprocessing can be used to identify the differences between the material element and the background). Well-known image processing techniques, such as dilation, thresholding, and contouring, can be used to identify the material element as different from the background. In process block 3505, segmentation can be performed. For example, the extracted information may include information relating to one or more material elements. Furthermore, a particular material element may be located on a seam of the conveyor belt when its image is extracted. Therefore, in this case, it may be necessary to isolate the image of a single material object from the background of the image. In the exemplary technique of process block 3505, the first step is to apply a high-contrast image; in this way, the background pixels are reduced to substantially all black pixels, and at least some pixels related to the material object are brightened to substantially all white pixels. The image pixels of the white material object are then enlarged to cover the entire size of the material object. After this step, the position of the material object is a high-contrast image of all white pixels on a black background. Then, a contour algorithm can be used to detect the boundaries of the material object. Boundary information is saved, and then the boundary positions are transferred to the original image. Then, segmentation is performed on the original image in areas larger than the previously defined boundaries. In this way, the material object is identified and separated from the background.

In optional process block 3506, material pieces can be transported along the conveyor system near the distance measuring device and/or sensing system to determine their size and/or shape. This may be useful if an XRF system, LIBS system, or some other spectral sensor is also implemented within the sorting system and such size and/or shape determination is required. In process block 3507, post-processing can be performed. Post-processing may involve adjusting the size of the captured information/data to prepare it for use in a neural network. This may also include modifying certain attributes (e.g., enhancing image contrast, changing the image background, or applying filters) to enhance the machine learning system's ability to classify material pieces. In process block 3509, the data size can be adjusted. In some cases, it may be necessary to adjust the data size to match the data input requirements of certain machine learning systems, such as neural networks. For example, neural networks may require image sizes much smaller than those captured by a typical digital camera (e.g., 225 x 255 pixels or 299 x 299 pixels). Furthermore, the smaller the amount of input data, the less processing time is required to perform classification. Therefore, a smaller data size can ultimately increase the throughput of the sorting system 100 and increase its value.

In process blocks 3510 and 3511, for each material part, the type or category of the material is identified/classified based on sensed/detected features. For example, process block 3510 can be configured with a neural network employing one or more machine learning algorithms that compare extracted features with features stored in a knowledge base generated during the training phase and assign the classification with the highest match to each material part based on this comparison. The algorithm of the machine learning system can process the extracted information/data in a hierarchical manner using automatically trained filters. The filters then respond with successful combinations in the next stage of the algorithm until a probability is obtained in the final step. In process block 3511, these probabilities can be used for each of the N categories to determine which of the N sorting bins the corresponding material item should be sorted into. For example, each of the N categories can be assigned to a sorting bin, and the material item under consideration is sorted into the bin corresponding to the category that returns the highest probability greater than a predetermined threshold. In this disclosed embodiment, such a predetermined threshold can be preset by the user. If no probability is greater than the predetermined threshold, the specific material item can be sorted into the outlier bin (e.g., sorting bin 140).

Next, in process block 3512, sorting devices corresponding to one or more categories of material items can be triggered. Between the time the image of the material item is captured and the time the sorting device is triggered, the material item has moved from the vicinity of the vision system and/or sensor system to a downstream position of the conveying system (e.g., at the conveying speed of the conveying system). In an embodiment of this disclosure, the triggering of the sorting device is timed such that when the material item passes through the sorting device mapped to the category of the material item, the sorting device is triggered, and the material item is transferred/discharged from the conveying system to its associated sorting bin. In this disclosed embodiment, the trigger of the sorting device can be timed by a corresponding position detector that detects when a material item passes before the sorting device and sends a signal to activate the trigger of the sorting device. In process block 3513, the sorting bin corresponding to the activated sorting device receives the transferred/discharged material items.

Figure 6 illustrates a flowchart of an exemplary embodiment of a process 400 for sorting material parts according to certain embodiments of the present disclosure. Process 400 can be configured to operate in any embodiment of the present disclosure described herein, including system 100 of Figure 1. Process 400 can be configured to operate in conjunction with process 3500. For example, according to certain embodiments of the present disclosure, process blocks 403 and 404 can be incorporated into process 3500 (e.g., operating in series or in parallel with process blocks 3503-3510) to combine the efforts of a vision system 110 implemented with a machine learning system with a sensor system (e.g., sensor system 120) not implemented with a machine learning system for classifying and/or sorting material parts.

The operation of process 400 can be performed by hardware and/or software, including within a computer system (e.g., computer system 3400 of Figure 8) that controls the sorting system (e.g., computer system 107 of Figure 1). In process block 401, a material is fed onto a transport system. Next, in optional process block 402, the material can be transported along the transport system near a distance measuring device and/or optical imaging system to determine the size and/or shape of the material. In process block 403, while the material has traveled near a sensing system, it can be amplified or stimulated with a type of energy suitable for the specific type of sensing technology used by the sensing system (e.g., a LIBS system). In process block 404, the physical properties of the material parts are sensed/detected by the sensor system. In process block 405, for at least some material parts, the type of the material is identified/classified based on (at least partially) the sensed/detected features, which can be combined with the classification of the machine learning system of vision system 110.

Next, if the materials are to be sorted, in process block 406, a sorting device corresponding to one or more categories of the materials is triggered. Between the time the materials are sensed and the time the sorting device is triggered, the materials have moved from the vicinity of the sensing system to a position downstream of the conveyor system at the conveying rate of the conveyor system. In some embodiments of this disclosure, the triggering of the sorting device is timed such that when a material passes through a sorting device mapped to a category of the material, the sorting device is triggered, and the material is transferred/discharged from the conveyor system to its associated sorting bin. In some embodiments of this disclosure, the triggering of the sorting device can be timed by a corresponding position detector that detects when a material item passes before the sorting device and sends a signal to activate the sorting device. In process block 407, the sorting bin corresponding to the activated sorting device receives the turned/discharged material items.

According to certain embodiments of this disclosure, multiple at least portions of system 100 can be linked together in a continuous manner to perform multiple iterations or sorting layers. For example, when two or more systems 100 are linked in this manner, the conveying system may be implemented with a single conveyor belt or multiple conveyor belts, conveying material pieces via a first vision system (and, according to some embodiments, a sensor system) configured to sort material pieces of a first group of heterogeneous mixture materials into a first group of one or more receptacles (e.g., sorting bins 136…139) via a sorter (e.g., a first automation control system 108 and associated one or more sorting devices 126…129), and then conveying the material pieces through a second vision system (and, according to some embodiments, another sensor system), which is configured to sort material pieces of a second group of heterogeneous mixture materials into a second group of one or more sorting bins via a second sorter.

This continuous system 100 can contain any number of such systems linked together in this manner. According to certain embodiments of this disclosure, each continuous visual system can be configured to separate materials that are different from those of the previous visual system.

According to various embodiments of this disclosure, different types or categories of materials can be classified using different types of sensors, each sensor being used in a machine learning system and combined to classify material components in waste or waste streams.

According to various embodiments of this disclosure, data from two or more sensors can be used in combination with one or more machine learning systems to perform material classification.

According to various embodiments of this disclosure, multiple sensor systems can be installed on a single transmission system, with each sensor system utilizing a different machine learning system. According to various embodiments of this disclosure, multiple sensor systems can be installed on different transmission systems, with each sensor system utilizing a different machine learning system.

Figures 7A-7B illustrate a system and process 1600 configured according to certain embodiments of the present disclosure for sorting multiple metal alloy parts. Figure 7A shows an exemplary non-limiting schematic diagram of such a system and process 1600 from the side view, while Figure 7B shows a top view. Although Figures 7A-7B depict three stages of sorting/separation, any number of such stages can be implemented according to various embodiments of the present disclosure.

Multiple metal alloy sheets 1601 can be conveyed (e.g., via conveyor belt 1602) to be picked up by an inclined conveyor system 1603. Note that for simplicity, material pieces 1601 are not depicted in Figure 7B. The conveyor system 1603 conveys material pieces 1601 through a sensor system 1610 for sorting and sorting. Any publicly available vision system 110 or sensor system 120 (e.g., LIBS, XRF, etc.) can be used.

In a non-limiting example, the material pieces 1601 supplied to the conveying system 1602 may be mixtures of aluminum alloys, including cast, forged, and/or extruded aluminum alloys of various alloy compositions. The AI system 1610 may be configured to identify, classify, and distinguish those material pieces made of forged aluminum alloys from those made of cast aluminum alloys. The conveying system 1603 may be configured to operate at a sufficient speed to “throw” material pieces classified as forged aluminum alloys onto a subsequent inclined conveying system 1604. Material pieces not classified as composed of forged aluminum alloys (e.g., cast and/or extruded alloys) are discharged by sorting device 1620 onto a lower-positioned conveying system 1606. For example, such a sorting device 1620 could be, for instance, an air jet nozzle as described herein, actuated to expel material parts not classified as forged aluminum alloys that have been "thrown" from the end of the conveying system 1603 onto the conveying system 1604 along a normal trajectory. Material parts not classified as forged aluminum alloys (e.g., cast and/or extruded alloys) could be conveyed into a box or container 1630, or they could be conveyed via another sensing system 120 as disclosed herein.

Material parts classified as forged aluminum alloys can be conveyed via an XRF or LIBS system 1611, which can be configured to identify, classify, and differentiate different forged aluminum alloys, including the same forged aluminum alloy family. A conveyor system 1604 can be configured to operate at a sufficient speed to "throw" material parts classified as belonging to one or more specific forged aluminum alloys onto a subsequent inclined conveyor system 1605. Other forged aluminum alloys can be discharged by a sorting device 1621 onto a lower-positioned conveyor system 1607. For example, such a sorting device 1621 could be an air jet nozzle as described herein, actuated to eject material pieces from the end of the conveying system 1604 onto the conveying system 1605 via a normal trajectory jet, sorting them into one or more specific forged aluminum alloys. The sorted material pieces can then be conveyed into a box or container 1631.

Material parts classified as belonging to one or more specific forged aluminum alloys can be transmitted through a sensor system 1612, which can be configured to identify and classify those material parts containing a critical amount of a specific material, in order to classify specific forged aluminum alloys known to contain such a specific material.

According to an alternative embodiment of this disclosure, cast aluminum alloys previously sorted by sorter 1620 can be transferred by conveyor system 1606 through an XRF system as described herein for classification/sorting of certain specific cast alloys. Cast aluminum alloy 319 has a single, observable large copper peak in its XRF spectrum, cast aluminum alloy 356 does not have such a large copper peak, while cast aluminum alloy 380 has large copper and zinc peaks. These significant differences can be utilized by the XRF system to perform high-precision sorting of these cast aluminum alloys. The classification/sorting of the casting portion is further disclosed in U.S. Patent Application No. 2021/0229133, which is incorporated herein by reference.

Conveyor systems 1605 and 1608 can be configured to operate in a similar manner to conveyor systems 1603 and 1604, sorter 1622 can be configured to operate in a similar manner to sorters 1620 and 1621, and bins 1632 and 1633 can be configured in a similar manner to bins 1630 and 1631.

It is important to note that system and process 1600 is not limited to a single conveyor system, but can be expanded to multiple production lines, each discharging sorted material parts onto multiple conveyor systems (e.g., conveyor systems 1606…1608). Similarly, one or more conveyor systems 1606…1608 can be implemented using any number of additional sensor systems to further sort these material parts.

Furthermore, the embodiments of this disclosure are not limited to classifying aluminum alloys, but can be configured to sort any number of different categories of materials, including but not limited to sorting various metals from Zorba (e.g., copper, brass, zinc, aluminum, etc.).

Referring now to Figure 8, a block diagram illustrating a data processing (“computer”) system 3400 is depicted, in which the configuration of embodiments of this disclosure can be implemented. (The terms “computer,” “system,” “computer system,” and “data processing system” are used interchangeably herein.) The configurations of computer system 107, automation control system 108, sensor system 120, and/or vision system 110 can be configured similarly to computer system 3400. Computer system 3400 may employ a local bus 3405 (e.g., a peripheral component interconnect (“PCI”) local bus architecture). Any suitable bus architecture can be used, such as Accelerated Graphics Port (“AGP”) and Industry Standard Architecture (“ISA”), etc. One or more processors 3415, volatile memory 3420, and non-volatile memory 3435 may be connected to a local bus 3405 (e.g., via a PCI bridge (not shown)). An integrated memory controller and cached memory may be coupled to one or more processors 3415. One or more processors 3415 may include one or more central processing units and/or one or more graphics processing units and/or one or more tensor processing units. Additional connections to the local bus 3405 may be made via direct component interconnects or via add-in boards. In the illustrated example, a communication (e.g., network (LAN)) adapter 3425, an I/O (e.g., a small computer system interface (“SCSI”) host bus) adapter 3430, and an expansion bus interface (not shown) may be connected to the local bus 3405 via direct component connection. An audio adapter (not shown), a graphics adapter (not shown), and a display adapter 3416 (coupled to a display 3440) may be connected to the local bus 3405 (e.g., via an add-on board inserted into an expansion slot).

User interface adapter 3412 can provide connectivity for keyboard 3413 and mouse 3414, modem (not shown), and additional memory (not shown). I/O adapter 3430 can provide connectivity for hard disk drive 3431, tape drive 3432, and CD-ROM drive (not shown).

An operating system can run on one or more processors 3415 and is used to coordinate and provide control over various components within the computer system 3400. In Figure 8, the operating system can be a commercially available operating system. An object-oriented programming system (e.g., Java, Python, etc.) can run alongside the operating system and provide calls to the operating system from one or more programs (e.g., Java, Python, etc.) executing on system 3400. Instructions for the operating system, the object-oriented operating system, and the programs can reside on a non-volatile memory storage device 3435, such as a hard disk drive 3431, and can be loaded into volatile memory 3420 for execution by the processor 3415.

Those skilled in the art will understand that the hardware in FIG8 may vary depending on the implementation. Other internal hardware or peripheral devices, such as flash ROM (or equivalent non-volatile memory) or optical disc drives, may be used to supplement or replace the hardware shown in FIG8. Furthermore, any process of this disclosure can be applied to a multiprocessor computer system or performed by multiple such systems 3400. For example, training of the vision system 110 may be performed by a first computer system 3400, while operating the vision system 110 for sorting may be performed by a second computer system 3400.

As another example, computer system 3400 may be a stand-alone system configured to boot without relying on any type of network communication interface, regardless of whether computer system 3400 includes any type of network communication interface. As another example, computer system 3400 may be an embedded controller configured with ROM and/or flash ROM to provide non-volatile memory for storing operating system files or user-generated data.

The examples depicted in Figure 8 and the examples described above are not intended to imply any architectural limitations. Furthermore, the various forms of computer programs disclosed herein can reside on any computer-readable storage medium used by a computer system (i.e., floppy disks, optical disks, hard disks, magnetic tapes, ROM, RAM, etc.).

As already described herein, embodiments of this disclosure can be implemented to perform the various functions described for identifying, tracking, classifying, and/or sorting materials. Such functions can be implemented in hardware and/or software, for example, within one or more data processing systems (e.g., data processing system 3400 of FIG. 8), such as the aforementioned computer system 107, vision system 110, sensor system 120, and/or automation control system 108. However, the functions described herein are not limited to implementation on any particular hardware/software platform.

As will be understood by those skilled in the art, various embodiments of this disclosure can be embodied as systems, processes, methods, and/or program products. Therefore, various embodiments of the invention can take the form of entirely hardware embodiments, entirely software embodiments (including firmware, resident software, microcode, etc.), or embodiments combining software and hardware embodiments, which are generally referred to herein as “circuit,” “electronic circuit,” “module,” or “system.” Furthermore, embodiments of this disclosure can take the form of program products embodied in one or more computer-readable storage media having computer-readable code embodied thereon. (However, any combination of one or more computer-readable media may be used. Computer-readable media may be computer-readable signal media or computer-readable storage media.)

Computer-readable storage media can be, for example, but not limited to, electronic, magnetic, optical, electromagnetic, infrared, biological, atomic, or semiconductor systems, devices, controllers, or apparatuses, or any suitable combination thereof, wherein the computer-readable storage media itself is not a transient signal. More specific examples of computer-readable storage media (not an exhaustive list) may include the following: an electrical connection having one or more wires, a portable computer floppy disk, a hard disk, random access memory (“RAM”) (e.g., RAM 3420 of FIG. 8), read-only memory (“ROM”) (e.g., ROM 3435 of FIG. 8), erasable programmable read-only memory (“EPROM” or flash memory), optical fiber, portable optical disc read-only memory (“CD-ROM”), optical storage device, magnetic storage device (e.g., hard disk drive 3431 of FIG. 8), or any suitable combination of the foregoing. In the context of this document, a computer-readable storage medium can be any tangible medium that can contain or store programs for use by or in connection with an instruction execution system, device, controller, or apparatus. Program code contained on a computer-readable signal medium can be transmitted using any suitable medium, including but not limited to wireless, wired, fiber optic, RF, etc., or any suitable combination of the foregoing.

Computer-readable signal media can include, for example, propagated data signals containing computer-readable program code in baseband or as part of a carrier. Such propagated signals can take any of a variety of forms, including, but not limited to, electromagnetic, optical, or any suitable combination thereof. Computer-readable signal media can be any computer-readable medium that is not computer-readable storage media and can communicate, propagate, or transmit programs for use by or in conjunction with an instruction execution system, device, controller, or apparatus.

The flowcharts and block diagrams in the figures illustrate the possible architecture, functionality, and operation of systems, methods, processes, and program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or code portion, comprising one or more executable programmatic instructions for implementing a specified logical function. It should also be noted that in some embodiments, the functions marked in the blocks may not appear in the order indicated in the diagram. For example, two blocks shown consecutively may actually execute substantially simultaneously, or these blocks may sometimes execute in reverse order, depending on the functions in question.

A module implemented in software for execution by various types of processors (e.g., GPU 3401, CPU 3415) can, for example, comprise physical or logical blocks of one or more computer instructions, which can be organized as objects, procedures, or functions. However, the executable file of the identified module does not need to be physically placed together; instead, it can include different instructions stored in different locations that, when logically combined, include the module and implement its stated purpose. In practice, an executable module can be a single instruction or multiple instructions, and can even be distributed across several different code segments, between different programs, and across multiple memory devices. Similarly, operational data (e.g., the material classification library described herein) can be identified and described within a module, and can be represented in any suitable form and organized in any suitable type of data structure. Operational data can be collected as a single dataset or distributed across different locations, including different storage devices. The data can be provided as electronic signals on a system or network.

These program instructions can be provided to one or more processors and/or controllers of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus (e.g., a controller) to produce a machine, such that the instructions, executed by the processor of the computer or other programmable data processing apparatus (e.g., GPU 3401, CPU 3415), establish circuits or devices for implementing functions/actions specified in one or more blocks of a flowchart and/or block diagram.

It should also be noted that each block and/or flowchart description in the block diagram, and combinations thereof, can be executed by a dedicated hardware-based system (e.g., which may include one or more graphics processing units (e.g., GPU 3401)) to perform a specified function or behavior, or a combination of dedicated hardware and computer instructions. For example, a module can be implemented as a hardware circuit, including a custom VLSI circuit or gate array, off-the-shelf semiconductors (e.g., logic chips, transistors, controllers, or other discrete components). Modules can also be implemented in programmable hardware devices, such as field-programmable gate arrays, programmable array logic, programmable logic devices, etc.

Computer program code, i.e., instructions, used to perform operations as disclosed herein, may be written in any combination of one or more programming languages, object-oriented programming languages including Java, Smalltalk, Python, C++, etc., traditional programming languages such as the "C" programming language or similar programming languages, programming languages such as MATLAB or LabVIEW, or any machine learning software disclosed herein. The program code may be executed entirely on the user's computer system, partially on the user's computer system as a standalone software package, partially on the user's computer system (e.g., a sorting computer system) and partially on a remote computer system (e.g., a computer system used to train a machine learning system), or entirely on a remote computer system or server. In the latter scenario, the remote computer system can be connected to the user's computer system via any type of network, including a local area network (“LAN”) or a wide area network (“WAN”), or can be connected to an external computer system (e.g., via the Internet provided by an Internet service provider). As an example of the foregoing, the various configurations of this disclosure can be configured to run on one or more of the configurations of computer system 107, automation control system 108, vision system 110, and sensor system 120.

These program instructions may also be stored in a computer-readable storage medium, other programmable data processing device, controller or other device that can instruct a computer system to operate in a particular manner so that the instructions stored in the computer-readable medium produce an article of work which includes instructions to implement one or more functions/actions specified in a flowchart and/or block diagram.

Program instructions may also be loaded onto a computer, other programmable data processing device, controller or other device to cause a series of operational steps to be executed on the computer, other programmable device or other device, thereby producing a computer-implemented process, such that the instructions executed on the computer or other programmable device provide a process for implementing the function/action specified in one or more blocks of the flowchart and/or block diagram.

One or more databases may be included on a host for storing and providing access to data for various implementations. Those skilled in the art will also understand that, for security reasons, any database, system, or component disclosed herein may include any combination of databases or components located in a single location or multiple locations, wherein each database or system may include any suitable security features, such as firewalls, access keys, encryption, decryption, etc. Databases can be of any type, such as relational, hierarchical, object-oriented, etc. Common database products available for implementing databases include IBM's DB2, any database products offered by Oracle Corporation, Microsoft Access from Microsoft Corporation, or any other database product. Databases may be organized in any suitable manner, including as data tables or lookup tables.

The association of certain data (e.g., for each scrap item processed by the sorting system described herein) can be achieved using any data association techniques known and practiced in the art. For example, the association can be performed manually or automatically. Automatic association techniques can include, for example, database retrieval, database merging, GREP, AGREP, SQL, etc. The association step can be accomplished using database merging functionality, for example, using key fields from each manufacturer and retailer table. Key fields partition the database based on the high-level category of the objects defined by the key fields. For example, a category can be designated as a key field in a first table and a second table, and then the two tables can be merged based on the category data in the key fields. In these implementations, the key fields in each merged table should ideally correspond to the same data. However, tables with similar but different data in their key fields can also be merged using, for example, AGREP.

This document refers to a device as "configured" or "configured to" perform certain functions. It should be understood that this can include selecting predefined logic blocks and logically associating them such that they provide specific logical functions, including monitoring or control functions. It can also include modifying the computer software-based logic processing of a control device, wiring discrete hardware components, or any or all of the above combinations. Such a configured device is physically designed to perform one or more specified functions.

Numerous specific details, such as programming examples, software modules, user choices, network transactions, database queries, database structures, hardware modules, hardware circuits, hardware chips, controllers, etc., are provided in this description to provide a comprehensive understanding of embodiments of this disclosure. However, those skilled in the art will recognize that this disclosure can be practiced without one or more specific details or with other methods, components, materials, etc. In other instances, well-known structures, materials, or operations may not be shown or described in detail to avoid obscuring the nature of this disclosure.

Throughout this specification, references to “implementation,” “multiple embodiments,” or similar language mean that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of this disclosure. Therefore, the phrases “in one embodiment,” “in one embodiment,” “multiple embodiments,” “certain embodiments,” “various embodiments,” and similar language throughout this specification may, but not necessarily, refer to the same embodiment. Furthermore, the features, structures, styles, and/or characteristics described in this disclosure may be combined in one or more embodiments in any suitable manner. Correspondingly, even if the initially claimed feature functions in some combinations, in some cases, one or more features in the claimed combination may be removed from the combination, and the claimed combination may derive to subcombinations or variations thereof.

The benefits, advantages, and solutions to the problems have been described above with reference to specific embodiments. However, the benefits, advantages, and solutions to the problems, and any elements that may cause any benefit, advantage, or solution to appear or become more apparent, should not be construed as key, essential, or fundamental features or elements of any or all of the claims. Furthermore, unless expressly described as necessary or key, no component described herein is essential for implementing this disclosure.

Those skilled in the art who have read this disclosure will recognize that changes and modifications can be made to the embodiments without departing from the scope of this disclosure. It should be understood that the specific embodiments shown and described herein are illustrative of this disclosure and its best mode, and are not intended to limit the scope of this disclosure in any way. Other variations may fall within the scope of the following patent applications.

Although this specification contains numerous details, these should not be construed as limiting the scope of this disclosure or its claims, but rather as descriptions of features specific to particular embodiments of this disclosure. The headings herein may not be intended to limit this disclosure, embodiments thereof, or other matters disclosed under those headings.

In this document, the term “or” may be intended to include, where “A or B” includes either A or B and also includes both A and B. As used herein, the term “and/or” when used in the context of an entity list refers to entities that exist individually or in combination. Thus, for example, the phrase “A, B, C and/or D” individually includes A, B, C, and D, but also includes any and all combinations and subcombinations of A, B, C, and D.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a,” “an,” and “the” may also be intended to include the plural forms unless the context clearly indicates otherwise.

All devices or steps within the scope of the following patent applications, along with the corresponding structures, materials, actions, and equivalents of functional elements, may be intended to include any structure, material, or action for performing a function in combination with elements of other claims as specifically asserted.

As used in this article with respect to identified attributes or conditions, "substantially" means a degree of bias that is small enough not to significantly impair the identified attribute or condition. In some cases, the permissible degree of accuracy bias may depend on the specific circumstances.

As used herein, multiple items, structural elements, constituent elements, and/or materials may be presented in a common list for convenience. However, these lists should be interpreted as if each member of the list were individually identified as a separate and unique member. Therefore, no single member of such a list should be interpreted as a factual equivalent to any other member of the same list solely based on its presentation in a common group, without any indication to the contrary.

Unless otherwise defined, all technical and scientific terms used herein (e.g., acronyms for chemical elements in the periodic table) have the same meaning as commonly understood by one of ordinary skill in the field to which this disclosure pertains. While any methods, apparatuses, and materials similar to or equivalent to those described herein may be used in the practice or testing of the currently disclosed subject matter, representative methods, apparatuses, and materials are now described.

Unless otherwise stated, all numerical values used in this specification and the scope of the patent application to indicate the quantity of ingredients, reaction conditions, etc., should be understood to be modified by the term “about” in all cases. Therefore, unless stated to the contrary, the numerical parameters described in this specification and the appended scope of the patent application are approximate values that may vary depending on the desired characteristics sought to be obtained from the subject matter of this disclosure. As used herein, when referring to values or quantities of mass, weight, time, volume, concentration, or percentage, the term “about” is intended to include variations of ±20% in some embodiments, ±10% in some embodiments, ±5% in some embodiments, ±1% in some embodiments, ±0.5% in some embodiments, and ±0.1% in some embodiments, because such variations are suitable for performing the disclosed method.

100: System 101: Material 103: Conveyor System 104: Conveyor Belt Motor 105: Position Detector 106: Sorter 107: Computer System 108: Automation Control System 109: Stationary or Live Camera 110: Visual or Optical Recognition System 111: Distance Measurement Device 112: Control System 120: Sensor System 121: Energy Source 122: Power Supply 123: Control System 124: Detector 125: Detector Electronics 126: Sorting Device 127: Sorting Device 128: Sorting Device 129: Sorting Device 136: Sorting Box 137: Sorting Box 138: Sorting Box 139: Sorting Box 140: Box 319: Cast Aluminum Alloy 356: Cast Aluminum Alloy 380: Cast Aluminum Alloy 400: Process 401: Block 402: Block 403: Block 404: Block 405: Block 406: Block 407: Block 1600: Process 1601: Metal Alloy Sheet 1602: Conveyor Belt 1603: Inclined Conveyor System 1604: Inclined Conveyor System 1605: Conveyor System 1606: Transmission System 1607: Transmission System 1608: Transmission System 1610: Sensor System 1611: Sensor System 1612: Sensor System 1620: Sorting Device 1621: Sorting Device 1622: Sorting Device 1630: Box 1631: Box 1632: Box 1633: Box 3400: Computer System 3401: GPU 3405: Local Bus 3412: User Interface Adapter 3413: Keyboard 3414: Mouse 3415: Processor 3416: Display Adapter 3420: Volatile Memory 3425: Communication Adapter 3430: I/O Adapter 3431: Hard Disk Drive 3432: Tape Drive 3435: Non-volatile Memory 3440: Display 3500: Process 3501: Block 3502: Block 3503: Block 3504: Block 3505: Block 3506: Block 3507: Block 3508: Block 3509: Block 3510: Block 3511: Block 3512: Block 3513: Block

[Figure 1] shows a schematic diagram of a sorting system configured according to an embodiment of the present disclosure.

[Figure 2] shows a visual image of an exemplary material part from cast aluminum.

[Figure 3] shows a visual image of an exemplary material part from an aluminum extrusion.

[Figure 4] shows a visual image of an exemplary material part from forged aluminum.

[Figure 5] illustrates a flowchart of the configuration according to the embodiments of this disclosure.

[Figure 6] illustrates a flowchart of the configuration according to the embodiments of this disclosure.

[Figures 7A and 7B] illustrate systems and processes for sorting materials according to certain embodiments of the present disclosure.

[Figure 8] shows a block diagram of a data processing system configured according to an embodiment of the present disclosure.

1600: Process

1601: Metal alloy sheet

1602: Conveyor Belt

1603: Inclined Transport System

1604: Inclined Transport System

1605: Transmission System

1606: Transmission System

1607: Transmission System

1608: Transmission System

1610: Sensor System

1611: Sensor System

1612: Sensor System

1620: Sorting Device

1621: Sorting Device

1622: Sorting Device

1630: Box

1631: Box

1632: Box

1633: Box

Claims (20)

An apparatus for processing a first mixture comprising multiple different material categories, the apparatus comprising: an image sensor configured to capture visual observation features of each of the materials in the first mixture; and a data processing system including a machine learning system implementing a neural network configured with a previously generated set of neural network parameters, classifying a first plurality of materials in the first mixture into a first category of materials based on the captured visual observation features, wherein the previously generated set of neural network parameters is uniquely associated with the first category of materials, wherein the plurality of materials in the first mixture classified into the first category of materials having a chemical composition different from that of the materials in the first mixture, and not classified into the first category of materials. According to the apparatus of claim 1, the previously generated set of neural network parameters uniquely associated with the first category of the material is generated from the visually observed features of one or more samples of the first category of the material. The apparatus according to claim 1, wherein the first category of material is cast aluminum alloy, further comprises: a first sorter configured to sort the first category of materials of the first mixture from the first mixture according to a function of a first plurality of materials of the first mixture, wherein sorting from the first mixture by the first sorter of the first plurality of materials of the first mixture produces a second mixture of materials, comprising the first mixture minus the first plurality of materials of the first mixture; a laser-induced breakdown spectroscopy (“LIBS”) system configured to classify a second plurality of materials of the second mixture into a second category of materials; and A second sorter, configured via a LIBS system according to a function of the classification of the second plurality of materials in the second mixture, is configured to sort a second plurality of materials of the second mixture from the second mixture, wherein the second mixture of materials includes forged aluminum material parts comprising a plurality of different forged aluminum alloys, and wherein the LIBS system is configured to classify certain parts of the second mixture as belonging to a first forged aluminum alloy, wherein the second sorter sorts certain parts of the second mixture from the second mixture according to the function of the classification of certain parts of the second mixture, wherein the sorting from the second mixture by the second sorter of certain parts of the classification produces a third mixture of materials, comprising the second mixture minus the certain parts from the second mixture, wherein the third mixture comprises materials belonging to a second forged aluminum alloy different from the first forged aluminum alloy. The apparatus according to claim 1, wherein the first category of material is cast aluminum alloy, further comprising: a first sorter configured to sort from the first mixture a first plurality of materials of the first mixture according to a function of the classification of the first plurality of materials of the first mixture; an X-ray fluorescence (“XRF”) system configured to classify a second plurality of materials of the first plurality of materials of the first mixture as belonging to a second category of materials according to a function of spectral data generated by the XRF system; and a second sorter configured, via the XRF system, to sort from the first plurality of materials of the first plurality of materials of the first mixture a second plurality of materials of the second mixture. The apparatus according to claim 1, wherein the previously generated set of neural network parameters is generated during a training phase, during which an artificial intelligence system implementing the neural network processes visual images of the first category of material control group representing materials. The apparatus according to claim 1, wherein the previously generated set of neural network parameters is designated to represent visually identifiable features that indicate the chemical composition of the material of the first category. A method for processing a first heterogeneous mixture of separable materials comprising multiple different types of materials, the method comprising: ... The LIBS system assigns a second classification to certain portions of the second heterogeneous mixture of materials, classifying them as belonging to a second type of material; and, according to a function of the second classification, sorts out certain portions of the second heterogeneous mixture of materials from the second heterogeneous mixture. According to the method of claim 7, the previously generated set of neural network parameters is generated from a previously generated classification of the first type of reference sample of the material. According to the method of claim 7, the sensor is a camera configured to capture visual images of each material piece of the first heterogeneous mixture of materials to generate image data, and wherein the captured features are visual observation features. According to the method of claim 7, wherein the first category of the material is a cast aluminum alloy, wherein the second heterogeneous mixture of the material comprises forged aluminum material parts containing multiple different forged aluminum alloys, and wherein the LIBS system is configured to classify certain portions of the second heterogeneous mixture as belonging to the first forged aluminum alloy, wherein a sorter sorts certain portions of the second heterogeneous mixture according to a function of the classification of certain portions of the second heterogeneous mixture. According to the method of claim 10, the sorting of materials produced by the sorter of a certain portion of the second heterogeneous mixture produces a third mixture of materials comprising the second heterogeneous mixture minus the second heterogeneous mixture, wherein the third mixture comprises materials belonging to a second forged aluminum alloy that is different from the first forged aluminum alloy. According to the method of claim 7, wherein the first category of the material is a cast aluminum alloy, wherein certain of the first heterogeneous mixture material results in a third heterogeneous mixture, the method further comprising: assigning a third classification to certain of the third heterogeneous mixture of the material by an XRF system, based on a function of spectral data generated by the XRF system, to belong to a third type of material; and sorting the certain of the third heterogeneous mixture of the material from the third heterogeneous mixture according to a function of the third classification. According to the method of claim 7, wherein the previously generated neural network parameter set is generated during a training phase, in which an artificial intelligence system implementing the neural network processes visual images of the first category of material control group representing materials. According to the method of claim 7, the previously generated set of neural network parameters is designated to represent visually identifiable features that indicate the chemical composition of the material of the first category. A computer program product stored on a computer-readable storage medium, when executed by a data processing system, includes the following steps: Using an artificial intelligence system implementing a neural network configured with a previously generated set of neural network parameters, assigning a first classification to certain portions of the first heterogeneous mixture of materials as belonging to a first type of material based on the characteristics of the extraction of each material component of the first heterogeneous mixture, wherein the previously generated set of neural network parameters is uniquely associated with the first type of material; Guiding the sorting of certain portions of the first heterogeneous mixture of materials from the first heterogeneous mixture according to a function of the first type, wherein the sorting produces a second heterogeneous mixture of materials comprising the first heterogeneous mixture of materials minus the sorted portions of the first heterogeneous mixture of materials; The second heterogeneous mixture received from the LIBS system contains certain materials belonging to a second category of a second type of material; and the second heterogeneous mixture contains certain materials that are sorted from the second heterogeneous mixture according to a function of the second category. The computer program product according to claim 15, wherein the previously generated set of neural network parameters is generated from a previously generated classification of a reference sample of the first type of material. The computer program product according to claim 15, wherein the captured feature is a visually observable feature captured by a camera. The computer program product according to claim 15, wherein the first category of material is cast aluminum alloy, wherein the second heterogeneous mixture of material comprises forged aluminum material parts containing multiple different forged aluminum alloys, and wherein the LIBS system is configured to classify certain portions of the second heterogeneous mixture as belonging to the first forged aluminum alloy, wherein the sorting of certain portions of the second heterogeneous mixture according to a function of the classification of certain portions of the second heterogeneous mixture is performed. The computer program product according to claim 15, wherein the previously generated set of neural network parameters is generated during a training phase, in which an artificial intelligence system implementing the neural network processes visual images of the first category of material control group representing materials. The computer program product according to claim 15, wherein the previously generated set of neural network parameters is designated to represent visually identifiable features that indicate the chemical composition of the material of the first category.