DE102024205924A1 - Method for the automated movement of a packaging unit in a production plant - Google Patents

Abstract

The invention relates to a method (100) for the automated movement of a packaging unit (2) in a production plant (1), comprising the following steps:
- Providing (101) an image of the packaging unit (2), wherein the image results from a capture by a camera (3),
- Performing (102) object detection based on the provided image to identify a type of packaging unit (2),
- Selecting (103) a specific gripper attachment (4) depending on the identified type of packaging unit (2),
- Moving (104) the packaging unit (2) using the selected specific gripper attachment (4).
Furthermore, the invention relates to a computer program, a device and a storage medium for this purpose.

Description

The invention relates to a method for the automated movement of a packaging unit in a production plant. Furthermore, the invention relates to a computer program, a device, and a storage medium for this purpose.

State of the art

The process of removing goods, or packaging units, from incoming packages is a crucial step in the logistics flow. There are generally two solutions to enable this process: either a highly defined package is provided, or the individual packaging units are removed manually.

Manual unpacking requires, in particular, the strict separation of the packaging materials into paper and plastic. However, identifying and handling these materials in the current production process necessitates human intervention.

Disclosure of the invention

The invention relates to a method with the features of claim 1, a computer program with the features of claim 8, a device with the features of claim 9, and a computer-readable storage medium with the features of claim 10. Further features and details of the invention will become apparent from the respective dependent claims, the description, and the drawings. Features and details described in connection with the method according to the invention naturally also apply in connection with the computer program, the device, and the computer-readable storage medium according to the invention, and vice versa, so that a reciprocal reference is always possible with regard to the disclosure of the invention.

The invention relates in particular to a method for the automated movement of a packaging unit in a production plant, comprising the following steps, wherein the steps can be performed repeatedly and/or sequentially. The production plant is in particular a surface mount technology production plant. The packaging unit is, for example, a printed circuit board, a component reel, in particular a surface mount device component reel, or a packaging material. The packaging material can be, for example, packing paper, bubble wrap, or polystyrene.

In a first step, an image of the packaging unit is preferably provided, the image being captured by a camera. The image is preferably captured from a top view of the packaging unit. The packaging unit can be located, for example, inside packaging such as a cardboard box or on a conveyor belt. It is also conceivable to use an ultrasonic, radar, lidar, or ultrasonic sensor as an alternative or additional method to a camera. It is also conceivable to provide at least two images of the packaging unit.

In a further step, object detection is preferably performed based on the provided image to identify the type of packaging unit. Alternatively or additionally, the packaging unit is classified based on the provided image.

For object detection and classification, a machine learning model, such as a convolutional neural network, is preferably used. Algorithms like Single Shot MultiBox Detector (SSD) or You Only Look Once (YOLO) are also conceivable. The machine learning model is specifically trained for classification and/or object detection. Training data can consist of images of various types of packaging units captured within a production facility. The training data can be labeled to indicate the respective packaging unit type. This training process results in a trained machine learning model that can be used for classification and/or object detection. Its use, and thus inference, is intended primarily within the production facility. The input data points can be pixels from image data or be based on them, enabling pixel-based classification and/or object detection. The input data can include sensor and/or image data, which at least partially result from acquisition with a sensor, preferably a camera sensor, and/or which have been at least partially synthesized, thus in particular replicating the real data of a sensor. Specifically, it can be provided that the values of image points, preferably pixels, of the image data represent a sensor's environment, in particular the packaging unit from a top view. A classification, preferably image classification and/or object detection, can be provided based on these values. This makes it possible, in particular, to identify the type of packaging unit. The classification can also take the form of semantic segmentation (i.e.,... h. a pixel- or area-based classification) and/or an object detection may be provided.

In a further step, a specific gripper attachment is preferably selected depending on the identified type of packaging unit. This can be done using rule-based methods, so that specific gripper attachments are assigned to and selected according to each type of packaging unit.

In a further step, the packaging unit is preferably moved using the selected specific gripper attachment. For example, the packaging unit can be moved from one conveyor belt to another, or in or out of packaging such as a carton, or from a carton into a container. The gripper attachment can, for example, be mounted on a robot arm after selection, particularly automatically, which is designed to move the packaging unit.

Another possibility is that, during object detection, a bounding box for the packaging unit is determined within a visual space, along with the unit's position. The packaging unit type is then identified based on classification information. Furthermore, the distance between the gripper attachment, or the system moving the gripper attachment, and the packaging unit can be estimated based on the packaging unit's position. This allows the packaging unit to then be gripped and moved based on its position. The classification information could be, for example, the packaging unit's color, shape, texture, or size.

It is also conceivable that a first type of packaging unit is a smooth surface without holes, and that the selected specific gripper attachment is a vacuum gripper. For example, the smooth surface could be a plastic surface, plastic packaging, or a smooth film.

• - Bestimmen eines Mittellochs der Bauteilrolle,
• - Greifen der Bauteilrolle, indem das aufblasbare Mittelstück in das Mittelloch der Bauteilrolle eingeführt und aufgeblasen wird.

Advantageously, the invention may provide that a second type of packaging unit is an uncovered component roll and, in the selection process, that the selected specific gripper attachment is a pneumatic gripper attachment with an inflatable center piece, and that the method further comprises the following steps:

• - Determining a center hole of the component roll,
• - Gripping the component roll by inserting the inflatable center piece into the center hole of the component roll and inflating it.

Determining the center hole can be performed as part of the object detection process. The classification information for the packaging unit could also include the position of the center hole within the packaging unit, which is linked, for example, to the corresponding packaging unit type.

• - Bestimmen einer genauen Form der Verpackungseinheit,
• - Zuweisen eines Aktionspunktes auf Basis der bestimmten genauen Form, wobei der Aktionspunkt eine zu greifende Stelle an der Verpackungseinheit für den Greifer-Aufsatz vorgibt,

A further advantage of the invention can be achieved if a third type of packaging unit has an irregular structure and the method further comprises the following steps:

• - Determining the exact shape of the packaging unit,
• - Assigning an action point based on the specified exact shape, where the action point defines a point to be gripped on the packaging unit by the gripper attachment,

The selection of the specific gripper attachment can then be further based on the precise shape of the packaging unit. An irregular structure is, in particular, a non-smooth structure, meaning that the packaging unit has raised areas and/or indentations. This could be, for example, packaging material such as filler paper. Based on the precisely defined shape and the assigned action point, the selected gripper attachment can engage at that point and grip the packaging unit to move it. An example of a gripper attachment for this third type would be a two-finger gripper or a parallel gripper.

• - Ausgeben einer Warnmeldung.

According to an advantageous embodiment of the invention, it can be provided that, in the event that no type of packaging unit can be identified during the object detection process, the method further comprises the following step:

• - Issuing a warning message.

The warning message can be displayed visually on a screen and/or audibly via a loudspeaker. The warning message may state that manual intervention by qualified personnel is necessary because the type of packaging unit cannot be identified.

The invention also relates to a computer program, in particular a computer program product, comprising instructions that are executed by a computer during the execution of the computer program. The computer causes the user to execute the method according to the invention. Thus, the computer program according to the invention offers the same advantages as those described in detail with reference to a method according to the invention.

The invention also relates to a data processing device configured to execute the method according to the invention. The device can, for example, be a computer that executes the computer program according to the invention. The computer can have at least one processor for executing the computer program. Alternatively, a non-volatile data storage device can be provided in which the computer program is stored and from which the computer program can be read by the processor for execution.

The invention may also relate to a computer-readable storage medium which contains the computer program according to the invention and/or includes instructions which, when executed by a computer, cause the computer to execute the method according to the invention. The storage medium is, for example, designed as a data storage device such as a hard drive and/or non-volatile memory and/or a memory card. The storage medium can, for example, be integrated into the computer.

Furthermore, the method according to the invention can also be implemented as a computer-implemented method. Alternatively or additionally, at least one of the disclosed method steps can be computer-implemented and/or carried out automatically.

• 1 eine schematische Visualisierung eines Verfahrens, einer Vorrichtung, eines Speichermediums sowie eines Computerprogramms gemäß Ausführungsbeispielen der Erfindung,
• 2 eine schematische Darstellung einer Produktionsanlage gemäß Ausführungsbeispielen der Erfindung.

Further advantages, features, and details of the invention will become apparent from the following description, in which exemplary embodiments of the invention are described in detail with reference to the drawings. The features mentioned in the claims and in the description can each be essential to the invention individually or in any combination. The drawings show:

• 1 a schematic visualization of a method, a device, a storage medium and a computer program according to exemplary embodiments of the invention,
• 2 a schematic representation of a production plant according to exemplary embodiments of the invention.

In 1 A method 100, a device 10, a storage medium 15 and a computer program 20 are schematically represented according to exemplary embodiments of the invention.

1 Figure 1 shows in particular an embodiment of a method 100 for the automated movement of a packaging unit 2 in a production plant 1. In a first step 101, an image of the packaging unit 2 is provided, the image resulting from a camera 3. In a second step 102, object detection is performed based on the provided image to identify the type of packaging unit 2. In a third step 103, a specific gripper attachment 4 is selected depending on the identified type of packaging unit 2. In a fourth step 104, the packaging unit 2 is moved using the selected specific gripper attachment 4.

2 Figure 1 schematically shows a production plant 1. This is preferably a surface mount technology production plant. Two packaging units 2 are schematically depicted within it. One of the packaging units 2 is a component roll 2a with a central hole 21a. For this purpose, a pneumatic gripper attachment 4a with an inflatable center piece 41a is preferably used during the selection process. The inflatable center piece 41a is inserted into the central hole 21a and inflated there to grip the component roll 2a. The other packaging unit 2 has, in particular, an irregular structure and is, for example, a packing paper. Here, the exact shape of the packaging unit 2 can first be determined in order to then assign it an action point 5. At this action point 5, the selected specific gripper attachment 4 preferably grips the packaging unit 2 to move it. An image of the respective packaging unit 2, 2a can be captured by a camera 3 to perform object detection.

The invention, according to the exemplary embodiments, relates in particular to the automation of the movement of packaging units in the electronics industry, wherein printed circuit boards and component reels of SMT components (both uncovered and vacuum-packed) can be the main targets of such a process. It is also particularly important to note that the packaging of such goods generally includes additional items for the safety of the goods during transport, such as plastic and paper packaging materials. For the same reason, the SMT component reels are preferably also packed in smaller container boxes. The method according to the exemplary embodiments can advantageously automate and facilitate the movement or removal of such packaging units as printed circuit boards, SMT reels (packaged, vacuum-packed, uncovered), and packaging material.

A camera can be mounted above a manual station in the production line to collect training data, taking a picture, for example, every two seconds. The packaging units in the images collected in this way serve, in particular, as a representative sample of different packaging configurations and the so-called supplier behavior.

Images can be captured of an open box or carton to reveal its contents. This allows the image processing application, particularly object detection, to categorize various packaging units, such as printed circuit boards, SMT reels (packaged, vacuum-packed, or uncovered), or packaging material, into categories or types, or to classify them as exceptions (i.e., unidentifiable categories/types). Furthermore, the current position of the packaging unit to be moved can be determined. Based on this data, a strategy can be defined for moving the packaging unit, the gripper attachment to be used, and the precise location to which the gripper attachment should be moved. The gripper attachments can be mounted or attached to a robot arm to perform the movement of the packaging unit.

Packaging units with a smooth surface can be gripped and moved using a vacuum gripper. This can include unloading cartons (vacuum-packed circuit boards or component reels, or uncovered component reels), moving paper or plastic packing material with a smooth surface, and moving closed or empty cartons.

It is particularly impossible to grip non-smooth surfaces, i.e., irregular structures such as paper or plastic packing material, with a vacuum gripper. In these cases, they can be moved using two-finger grippers or other mechanical aids (without a robot arm), for example, after object detection, i.e., removed from a conveyor belt.

When gripping objects with a smooth but textured surface (e.g., uncovered component reels), there is a particular risk that they will fall off after being gripped if moved at high acceleration, which can lead to production rejects. To prevent this, a pneumatic gripper attachment can be used, which locks the object in place through both shape and force. The gripper attachment can use a central hole of the component reel as a clamping point. The pneumatic gripper attachment preferably fits into this central hole.

These types of gripper attachments advantageously enable the movement of various packaging units without human intervention in an automated process. Preferably, there is also an exception process in which skilled human personnel can be notified and deployed for the movement. In this case, packaging units that must not be moved in the automated process are moved manually.

The invention, according to the exemplary embodiments, relates in particular to an automation process for removing and unpacking packaging units. However, the idea according to the invention can also be used for other processes involving the separation of different objects.

The method according to exemplary embodiments of the invention can comprise the following steps. In a first step, preferably an opened package, in particular a carton, is fed in. In a second step, an image of the contents of the opened package can be captured from above. In a third step, an AI-based method, i.e., object detection, can be performed, whereby a subsequent packaging unit within the package is nominated or classified based on the captured image. The object detection outputs, in particular, a regressed 2D bounding box in the image space, a 3D distance estimate, and classification information. In a fourth step, based on the outputs of the previous step and defined rules, a suitable, specific gripper attachment can be selected that can grasp and move the packaging unit or remove it from the carton. If the packaging unit has a flat surface without holes (e.g., printed circuit boards, SMT reels vacuum-packed, or SMT reels packaged in flat packaging, empty container boxes), a vacuum gripper is preferably selected. If the packaging unit is an uncovered component reel, which typically has holes on its surface, making the use of conventional vacuum grippers inefficient, the following procedure is preferably used. During object detection, a central hole of the component reel can be determined, and a corresponding coordinate assignment can be used. Subsequently, a specially designed inflatable gripper attachment can be used to grasp the packaging unit by positioning its inflatable center section at the central hole of the component reel. If the packaging unit has a If the packaging unit has an irregular structure, e.g., packaging material such as crumpled paper, etc., the following procedure can be used. During object detection, or in a subsequent object detection step, the exact shape of the packaging unit can be determined. An action point can then be assigned based on this determined shape. The appropriate gripper attachment, e.g., a parallel gripper, can then be selected. If the packaging unit is an exception, i.e., if no packaging unit type can be identified, a flag for exception handling is preferably set; that is, a corresponding notification is issued, and the system waits for a specialist to resolve the issue. In a further step, the packaging unit is preferably moved using the selected gripper attachment. For example, the packaging unit can be removed from its packaging. The packaging unit can then be placed in a predefined target position, e.g., in another container. The process can then begin again with the second step until the packaging is empty. Afterward, the empty packaging can be removed from a conveyor belt. The process can then be repeated with a new package.

The preceding explanation of the embodiments describes the present invention solely by way of examples. Naturally, individual features of the embodiments can be freely combined with one another, provided this is technically feasible, without departing from the scope of the present invention.

Claims (10)

Method (100) for the automated movement of a packaging unit (2) in a production plant (1), comprising the following steps: - Providing (101) an image of the packaging unit (2), wherein the image results from a camera (3) capture, - Performing (102) object detection based on the provided image to identify a type of packaging unit (2), - Selecting (103) a specific gripper attachment (4) depending on the identified type of packaging unit (2), - Moving (104) the packaging unit (2) using the selected specific gripper attachment (4). Procedure (100) according to Claim 1 , characterized in that , during the execution (102) of the object detection, a boundary frame for the packaging unit (2) in an image space and a position of the packaging unit (2) are determined and the type of the packaging unit (2) is determined on the basis of classification information. Method (100) according to one of the preceding claims, characterized in that the production plant (1) is a Surface Mount Technology production plant and the packaging unit (2) is a printed circuit board, a component reel (2a), in particular a Surface Mount Device component reel, or a packaging material. Method (100) according to one of the preceding claims, characterized in that a first type of packaging unit (2) is a smooth surface without holes and, in the context of selection, the selected specific gripper attachment (4) is a vacuum gripper. Method (100) according to one of the preceding claims, characterized in that a second type of packaging unit (2) is an uncovered component roll (2a) and, in the context of selecting the specific gripper attachment (4), is a pneumatic gripper attachment (4a) with an inflatable center piece (41a), and the method (100) further comprises the following steps: - determining a central hole (21a) of the component roll (2a), - gripping the component roll (2a) by inserting the inflatable center piece (4a) into the central hole (21a) of the component roll (2a) and inflating it. Method (100) according to one of the preceding claims, characterized in that a third type of packaging unit (2) is an irregular structure and the method (100) further comprises the following steps: - determining an exact shape of the packaging unit (2), - assigning an action point (5) based on the determined exact shape, wherein the action point (5) specifies a gripping location on the packaging unit for the gripper attachment (4), wherein the selection (103) of the specific gripper attachment (4) is further carried out based on the determined exact shape of the packaging unit (2). Method (100) according to one of the preceding claims, characterized in that, in the event that no type of packaging unit (2) can be identified during the execution (102) of the object detection, the method (100) further comprises the following step: - issuing a warning message. Computer program (20), comprising instructions which, when the computer program (20) is executed by a computer (10), cause it to execute the method (100) according to one of the preceding claims. Device (10) for data processing, which is configured to perform the method (100) according to one of the Claims 1 until 7 to execute. Computer-readable storage medium (15), comprising instructions which, when executed by a computer (10), cause it to perform the steps of the procedure (100) according to one of the Claims 1 until 7 to execute.