WO2024232680A1 - Inspection system and inspection method for inspecting low density foreign substance - Google Patents

Abstract

According to an embodiment of the present disclosure, provided is a foreign substance inspection system comprising: a conveyor belt configured to transfer a product to be inspected; a product detection sensor for detecting introduction of the product to be inspected to the conveyor belt; a PLC that generates a camera device trigger signal and transmits same to a camera device when the introduction of the product to be inspected is detected by the product detection sensor; the camera device configured to capture an image of the product to be inspected and process the captured image; and a PC configured to receive a deep learning model trained by a server and output prediction of whether a foreign substance is included by inputting the image processed by the camera device to the deep learning model, wherein the camera device comprises an MWIR camera for photographing the product to be inspected by using a wavelength of 3 ㎛ to 5 ㎛.

Description

Inspection system and method for inspecting low-density foreign matter

The present disclosure relates to an inspection system and an inspection method for inspecting low-density foreign substances, and more particularly, to an inspection system and an inspection method capable of distinguishing between normal products and foreign substances with high accuracy by combining thermal imaging and machine vision.

The material described in this section merely provides background information for the present disclosure and does not constitute prior art.

Recently, as the number of single-person households increases and tastes for convenience foods increase, the production of easy-to-cook foods is increasing.

Meanwhile, despite thorough hygiene management, various foreign substances can be introduced during the manufacturing stage of easy-to-cook foods. Since consumers are more averse to foreign substances being introduced into food than other product groups, thorough foreign substance inspection is required for food groups before product shipment. Various technologies have been proposed to inspect for the introduction of foreign substances.

For example, according to the KR 10-2507591 B1 document, a product can be inspected for foreign matter using an X-ray image. When using an X-ray image, the product to be inspected and the foreign matter are displayed differently depending on the difference in density.

However, in the case of X-ray images, there is a problem in that it is difficult to identify low-density foreign substances that have entered the product being inspected. Here, low-density foreign substances include vinyl, plastic, etc.

Meanwhile, cameras that utilize the wavelength band of SWIR (Short Wavelength InfraRed) or LWIR (Long Wavelength InfraRed) are also used to detect low-density foreign substances, but there is a problem that waves in these wavelength bands cannot penetrate the packaging material.

In addition, conventional foreign substance inspection systems have the problem that even if they detect defective products containing foreign substances, they are difficult to detect if defective products are mixed in with normal products.

Accordingly, the present disclosure aims to provide a foreign substance inspection system and inspection method capable of identifying low-density foreign substances and photographing them through packaging materials.

In addition, the present disclosure aims to provide a foreign substance inspection system and inspection method capable of preventing the problem of defective products being mixed among normal products.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

According to one embodiment of the present disclosure, a foreign matter inspection system is provided, including: a conveyor belt configured to transport a product to be inspected; a product detection sensor configured to detect the introduction of the product to be inspected onto the conveyor belt; a PLC configured to generate a camera device trigger signal and transmit the signal to the camera device when the introduction of the product to be inspected is detected by the product detection sensor; the camera device configured to photograph the product to be inspected and process the photographed image; and a PC configured to receive a deep learning model learned by a server and input the image processed by the camera device into the deep learning model, thereby outputting a predicted presence or absence of a foreign matter; wherein the camera device includes an MWIR camera that photographs the product to be inspected using a wavelength of 3 ㎛ to 5 ㎛; and an image processing unit configured to process a raw image photographed by the MWIR camera.

In addition, preferably, the server of the foreign matter inspection system according to one embodiment of the present disclosure includes: a thermal image input module configured to receive a raw image captured by a MWIR camera; a foreign matter inclusion input module configured to receive information on whether a product to be inspected contains a foreign matter; an image processing unit configured to process the input raw image to generate a processed image; a learning information storage unit configured to generate and store a learning information dataset using the processed image and the foreign matter inclusion; and a learning unit configured to learn a deep learning model using the processed image as input data and the foreign matter inclusion as output data.

In addition, preferably, the image processing unit of the foreign matter inspection system according to one embodiment of the present disclosure binarizes the raw image and generates a processed image by converting the binarized image into a preset size, and when the PC receives the image processed by the image processing unit, it outputs a predicted foreign matter inclusion of 0 or 1.

In addition, preferably, in the foreign matter inspection system according to one embodiment of the present disclosure, if the predicted foreign matter inclusion is 0, the PC displays the raw image or the processed image through the output unit, and stores the raw image or the processed image of the product to be inspected, metadata of the raw image, and a value of 0 as a data set in a database.

In addition, preferably, in the foreign matter inspection system according to one embodiment of the present disclosure, if the predicted foreign matter inclusion is 1, the PC transmits 1 to the PLC, the PLC waits for a predetermined time, generates a lighting signal and transmits the lighting signal to a warning light, the PLC sets a counter 1 in a memory, generates a rejector operation signal and transmits the rejector operation signal to a rejector, and when the rejector receives the rejector operation signal, moves the inspection target product corresponding to the predicted foreign matter inclusion to a defective product loading space.

In addition, preferably, the defective product loading space of the foreign substance inspection system according to one embodiment of the present disclosure is provided with a defective product sensor configured to detect the introduction of the moved inspection target product, and when the PLC receives a detection signal of the defective product sensor within a preset time from the time when counter 1 is set in the memory, the PLC clears the memory, and when the PLC does not receive a detection signal of the defective product sensor within a preset time from the time when the memory counter 1 is set, the PLC generates a conveyor belt stop signal and transmits the conveyor belt stop signal to the conveyor belt.

In addition, according to one embodiment of the present disclosure, a method for inspecting foreign substances is provided, including: (a1) a step in which a deep learning model is learned by a server, which uses a processed image of a product to be inspected as input data and whether or not a foreign substance is included as output data; (a2) a step in which the deep learning model is transmitted to a PC; (b1) a step in which the introduction of a product to be inspected is detected by a product detection sensor, and the product detection sensor generates a product introduction signal and transmits the signal to a PLC; (b2) a step in which the PLC receives the product introduction signal, waits for a predetermined time, and generates a trigger signal and transmits the signal to a camera device; (b3) a step in which the camera device captures a raw image of the product to be inspected using a MWIR camera, and the camera device processes the image using an image processing unit; (b4) a step in which the processed image is transmitted to the PC; and (c) a step in which, when the processed image is input to the deep learning model, a predicted foreign substance inclusion or not is output; wherein the MWIR camera captures the product to be inspected using a wavelength of 3 ㎛ to 5 ㎛.

In addition, preferably, the step (a1) of the inspection method according to one embodiment of the present disclosure includes: (a10) a step in which the thermal image input module of the server receives a raw image captured using a MWIR camera; (a11) a step in which the foreign matter inclusion input module of the server receives an input of whether a foreign matter is included corresponding to the raw image captured in the step (a10); (a12) a step in which the image processing unit of the server processes the raw image input in the step (a10) to generate a processed image; and (a13) a step in which the learning unit of the server trains a deep learning model that uses the processed image generated in the step (a12) as input data and the foreign matter inclusion input in the step (a11) as output data.

In addition, preferably, the step (c) of the inspection method according to one embodiment of the present disclosure includes: (c10) a step of outputting the predicted presence or absence of foreign matter as 0; (c11) a step of the PC displaying the raw image or the processed image through an output unit; and (c12) a step of the PC transmitting and storing the raw image or the processed image of the product to be inspected, metadata of the raw image, and a value of 0 as a dataset in a database.

In addition, preferably, the step (c) of the inspection method according to one embodiment of the present disclosure further includes: (c20) a step in which the predicted foreign substance inclusion status is output as 1; (c21) a step in which the PC transmits 1 to the PLC; (c22) a step in which the PLC waits for a predetermined time and then generates a lighting signal and transmits the lighting signal to a warning light; (c23) a step in which the PLC sets a counter 1 in a memory and generates a rejector operation signal to generate the rejector operation signal in the rejector; and (c24) a step in which, when the rejector receives the rejector operation signal, the rejector moves the inspection target product corresponding to the predicted foreign substance inclusion status to a reject loading space.

In addition, preferably, the defective product loading space of the inspection method according to one embodiment of the present disclosure is provided with a defective product sensor configured to detect the introduction of the moved inspection target product, and the step (c) further includes: (c25) a step of the PLC clearing the memory if the PLC receives a detection signal of the defective product sensor within a preset time from a point at which counter 1 is set in the memory; and (c26) a step of the PLC generating a conveyor belt stop signal and transmitting the conveyor belt stop signal to the conveyor belt if the PLC does not receive a detection signal of the defective product sensor within a preset time from a point at which the memory counter 1 is set.

As described above, the foreign substance inspection system and inspection method according to one embodiment of the present disclosure photograph a product to be inspected by irradiating a wavelength of an energy band capable of detecting low-density foreign substances while being able to penetrate packaging materials, thereby enabling detection of foreign substances that have entered a product to be inspected after packaging is completed, thereby increasing the probability of detecting foreign substances.

In addition, the foreign substance inspection system according to one embodiment of the present disclosure has the effect of properly notifying the user of the need to remove defective products by stopping all operations if defective products are not properly removed.

FIG. 1 is a block diagram of a foreign substance inspection system according to one embodiment of the present disclosure.

FIG. 2 is a block diagram of a PC according to one embodiment of the present disclosure.

FIG. 3 is a flowchart of a foreign substance inspection method according to one embodiment of the present disclosure.

FIG. 4 is a flowchart of a foreign substance inspection method according to one embodiment of the present disclosure.

FIG. 5 is a block diagram of a server according to one embodiment of the present disclosure.

FIG. 6 is a flowchart of a method for learning a deep learning model according to one embodiment of the present disclosure.

FIG. 7 compares a foreign substance inspection system according to one embodiment of the present disclosure with a conventional inspection system.

Hereinafter, some embodiments of the present disclosure will be described in detail with reference to exemplary drawings. When adding reference numerals to components of each drawing, it should be noted that the same components are given the same numerals as much as possible even if they are shown in different drawings. In addition, when describing the present disclosure, if it is determined that a specific description of a related known configuration or function may obscure the gist of the present disclosure, the detailed description thereof will be omitted.

In describing components of embodiments according to the present disclosure, symbols such as first, second, i), ii), a), b), etc. may be used. These symbols are only for distinguishing the components from other components, and the nature or order or sequence of the components is not limited by the symbols. When a part in the specification is said to "include" or "provide" a component, this does not mean that other components are excluded, but rather that other components can be further included, unless explicitly stated otherwise.

In the present disclosure, the “product to be inspected” is a product inspected by a foreign matter inspection system, and may preferably refer to packaging materials and food within the packaging materials. In the present disclosure, the product to be inspected is described as packaging materials and food within the packaging materials, but is not necessarily limited thereto.

1. Description of the foreign substance inspection system

FIG. 1 is a block diagram of a foreign substance inspection system according to one embodiment of the present disclosure. FIG. 2 is a block diagram of a PC according to one embodiment of the present disclosure.

Referring to FIGS. 1 and 2, a foreign matter inspection system (1) according to one embodiment of the present disclosure is described. The foreign matter inspection system (1) is configured to detect foreign matters introduced into a product to be inspected and to distinguish between normal products and defective products. To this end, the foreign matter inspection system (1) includes all or part of a PLC (10), a defective product sensor (11), a rejector (12), a conveyor belt (13), a warning light (14), a camera device (15), a product detection sensor (16), a PC (17), a server (18), and a database (19).

A PLC (programmable logic controller, 10) is configured to control the overall system (1) by communicating with other components included in the foreign substance inspection system (1) via wired or wireless means.

The defective product sensor (11) is configured to sense at least a portion of the inspection target product placed on the conveyor belt (13), specifically, the inspection target product determined to be a defective product.

The conveyor belt (13) is formed to transport the product to be inspected. The product to be inspected is fed into one end of the conveyor belt (13), and the product to be inspected is discharged from the other end. At this time, a driving unit is provided on the conveyor belt (13), so that the conveyor belt (13) can transport the product to be inspected in one direction.

The inspection target product transported on the conveyor belt (13) is determined as a normal product or a defective product. At this time, the inspection target product determined as a defective product can be taken out of the conveyor belt (13) by the rejector (12). For this purpose, the rejector (12) may be, for example, a take out robot (not shown), but is not necessarily limited thereto.

The warning light (14) lights up for a preset period of time when a defective product extracted by the rejector (12) is sensed by the defective product sensor (11).

The camera device (15) is configured to photograph a product to be inspected being transported by a conveyor belt (13) and to process the photographed image. To this end, the camera device (15) includes a MWIR camera (150) and an image processing unit (152).

The MWIR camera (150) is configured to photograph the product to be inspected. The MWIR camera (150) photographs the product to be inspected using mid-wave infrared (MWIR). At this time, MWIR means a wavelength of 3 to 5 ㎛. While a conventional SWIR band camera generally cannot penetrate packaging materials formed of materials including NY, PE, and LDPE, the MWIR camera (150) using the wavelength of the MWIR band can penetrate packaging materials formed of materials including NY, PE, and LDPE, thereby enabling detection of foreign substances introduced into the packaging material after packaging is completed. As a result, foreign substances introduced during packaging can be detected, thereby increasing the probability of detecting foreign substances. In other words, the MWIR camera (150) can detect radiant heat of the product to be inspected.

The image processing unit (152) is configured to process a raw image captured by the MWIR camera (150).

Specifically, the image processing unit (152) generates a black and white image by binarizing a raw image and converting it into gray scale. Thereafter, the image processing unit (152) generates a processed image by converting the generated black and white image into an image of a preset size. Here, the preset size may be, for example, 223 pixels.

The image processed by the image processing unit (152) is transmitted to the PC (17).

The product detection sensor (16) is arranged at one end of the conveyor belt (13), i.e., at the location where the product to be inspected is introduced, and is configured to detect the introduction of the product. When the product detection sensor (16) detects a product, it can generate a detection signal and transmit it to the PLC (10).

The PC (17) receives a deep learning model learned by the server (18) and is configured to input a processed image transmitted by the camera device (15) into the deep learning model. The PC (17) is configured to output whether or not a foreign substance is included corresponding to the input processed image.

To this end, the PC (17) includes all or part of an input unit (170), an output unit (171), a control unit (172), a processing unit (173), and a communication unit (174) (see FIG. 2). Meanwhile, the components of the PC (17) described in the present disclosure only describe the configurations necessary for inspection according to the present disclosure, and descriptions of additional components typically included in the PC (17) are omitted.

The input unit (170) is configured to receive commands or data from the outside.

A command or data input through the input unit (170) can be transmitted to another component of the PC (17). In the present disclosure, an image processed by the image processing unit (152) can be input through the input unit (170).

The output unit (171) is configured to output a command or data processed or generated by the PC (17). In the present disclosure, the output unit (171) may include a display and a speaker.

The control unit (172) is configured to control one or more other components of the PC (17). Here, the one or more other components are concepts including both hardware and software. That is, the control unit (172) can control the input unit (170), the output unit (171), the processing unit (173), and the communication unit (174) of the PC (17).

The processing unit (173) can perform a data processing or calculation function, and as part of the data processing or calculation function, is configured to store commands or data received from other components in volatile memory, process the commands or data stored in the volatile memory, and store the results in non-volatile memory.

The processing unit (173) can input the processed image input through the input unit (170) into a deep learning model downloaded in advance to calculate whether or not a foreign substance is included. Meanwhile, a detailed description of the deep learning model is described in FIGS. 5 and 6.

The communication unit (174) is configured for communication with external devices, and through the communication unit (174), the PC (17) can send and receive commands and data with the PLC (10), camera device (15), server (18), and database (19).

PC (17) classifies the result value as "1" when the product to be inspected contains foreign substances, i.e., when it is judged to be a defective product. In addition, PC (17) classifies the result value as "0" when the product to be inspected does not contain foreign substances, i.e., when it is a normal product.

If judged as 1, the PC (17) transmits a predetermined signal to the PLC (10). At this time, the PC (17) and the PLC (10) can send and receive signals using the SSR or Ethernet/IP Interface.

If judged as 0, the PC (17) can provide the user with an image captured by the camera device (15) or a processed image and an OK display through the output unit (171). In addition, the PC (17) can store the image of the inspection target product judged as 0, the metadata of the image, and the 0 value in the database (19).

The server (18) is configured to process, store, and transmit/receive data. The server (18) is configured to train a deep learning model. In this regard, it is described in detail in FIGS. 5 and 6. Meanwhile, in the present disclosure, the server (18) may be a separately provided device such as an IDC, but is not necessarily limited thereto, and may be a cloud.

The database (19) is configured to temporarily or permanently store data received from the PC (17). The database (19) may be a separate storage device provided outside the PC (17), but is not necessarily limited thereto, and may be a storage device provided in the PC (17) or PLC (10).

2. Description of foreign substance inspection method

2.1. Description of method for determining whether foreign substances are included

FIG. 3 is a flowchart of a foreign substance inspection method according to one embodiment of the present disclosure.

Referring to Figure 3, the process from the step of introducing the product to be inspected into the foreign substance inspection system (1) to the step of determining whether or not it contains foreign substances is described.

When a product to be inspected is introduced into the foreign substance inspection system (1), the introduction of the product to be inspected is detected by the product detection sensor (16). The product detection sensor (16) generates a product introduction signal and transmits it to the PLC (10).

When the PLC (10) receives a product introduction signal, it waits for a first delay time, then generates a trigger signal for shooting and transmits it to the camera device (15). Here, the first delay time is preferably the same time as the time it takes for the introduced inspection target product to be transported to the location where the camera device (15) is placed.

When the camera device (15) receives a trigger signal, it can detect the radiant heat of the product to be inspected by photographing the product to be inspected using the MWIR camera (150). The raw image photographed by the MWIR camera (150) is processed by the image processing unit (152) of the camera device (15).

The image processing unit (152) binarizes the raw image and converts it to gray scale to generate a black and white image. Thereafter, the image processing unit (152) generates a processed image by converting the generated black and white image into an image of a preset size. Here, the preset size may be, for example, 223 pixels.

The image processed through the above process is transmitted to the PC (17).

At this time, the PC (17) can download a deep learning model from the server (18) in advance and store it in internal memory (not shown) before receiving the processed image.

PC (17) can input the processed image into the aforementioned deep learning model and output whether or not a foreign substance is included. At this time, PC (17) can output the result value as 1 when a foreign substance is included and output the result value as 0 when a foreign substance is not included.

2.2. Description of the method after determining whether foreign substances are included

FIG. 4 is a flowchart of a foreign substance inspection method according to one embodiment of the present disclosure.

Referring to Fig. 4, the steps after the presence or absence of a foreign substance is determined by the foreign substance inspection system (1) are described.

If it is determined by the PC (17) that there is no foreign substance, i.e., if the result value is 0, the PC (17) outputs a thermal image and an OK indication together to the output unit (171). Here, the thermal image may be an image captured by the camera device (15) or a processed image.

In addition, the PC (17) can store the image of the inspection target product judged as 0, the metadata of the image, and the 0 value in the database (19). At this time, the data stored in the database (19) can be used for learning or updating the deep learning model in the future.

Meanwhile, if it is determined by the PC (17) to contain a foreign substance, i.e., if the result value is 1, the PC (17) transmits the result value to the PLC (10). At this time, the PC (17) and the PLC (10) can transmit and receive signals or data using the SSR or Ethernet/IP Interface.

When the PLC (10) receives the result value 1 from the PC (17), it waits for the second delay time. Here, the second delay time may be the same as the time taken for the inspection target product determined to be defective to be transported from the location where the camera device (15) is located to the location where the rejector (12) is located.

The PLC (10) generates a lighting signal and transmits it to the warning light (14). When the warning light (14) receives the lighting signal, it lights up for a preset period of time. This allows the user to know that there are defective products among the products to be inspected.

Simultaneously with the generation of the lighting signal, or sequentially, the PLC (10) sets counter 1 in the memory (not shown), generates a rejector operation signal, and transmits it to the rejector (12).

The rejector (12) operates when it receives a rejector operation signal. The rejector (12) can move a product to be inspected that has been determined to be defective from the conveyor belt (13) to a space provided separately. For this purpose, the rejector (12) may be a take-out robot, but is not necessarily limited thereto.

A defective product sensor (11) is provided in a separately prepared space. At this time, when a product to be inspected is detected by the defective product sensor (11), the defective product sensor (11) generates a product detection signal and transmits it to the PLC (10).

PLC (10) checks whether a product detection signal is received within a preset time from the time counter 1 is set in the memory.

At this time, if a product detection signal is received within a preset time, the PLC (10) clears the memory. That is, if a defective product is detected and the defective product is detected by the defective product sensor (11) within a certain time, the PLC (10) determines that the defective product has been removed from the conveyor belt (13), and can continue performing the existing work.

Meanwhile, if a product detection signal is not received within a preset time, the PLC (10) generates a conveyor belt stop signal and transmits it to the conveyor belt (13).

When the conveyor belt (13) receives a conveyor belt stop signal, it stops operating. That is, when a defective product is detected and the defective product is not detected by the defective product sensor (11) within a certain period of time, the PLC (10) determines that the defective product has not been properly removed from the conveyor belt (13), and stops all operations so that the user can be notified of the need to remove the defective product.

3. Description of the learning method of the deep learning model

FIG. 5 is a block diagram of a server according to one embodiment of the present disclosure.

Referring to FIG. 5, the server (18) includes all or part of an input unit (180), an image processing unit (183), a learning information storage unit (184), and a learning unit (185).

The input unit (180) is configured to receive data for learning. The input unit (180) includes a thermal image input module (181) and a foreign substance inclusion input module (182).

The thermal image input module (181) is configured to receive a raw image as a thermal image captured by the MWIR camera (150).

The input module (182) for detecting whether a foreign substance is included is configured to receive input on whether a foreign substance is included inside the product to be inspected, captured by the MWIR camera (150).

The image processing unit (183) is configured to preprocess the image input by the thermal image input module (181). Specifically, the image processing unit (183) binarizes a raw image and converts it to gray scale to generate a black and white image.

Thereafter, the image processing unit (183) generates a processed image by converting the generated black and white image into an image of a preset size. Here, the preset size may be, for example, 223 pixels.

Through the image processing process described above, information used for learning can be transformed into a shape and size appropriate for learning.

The learning information storage unit (184) stores the processed image and whether or not the processed image contains foreign substances as a dataset. At this time, when the learning information storage unit (184) converts the processed image into a dataset, the RAY or NUMPY library can be used to create a dataset by associating the processed image and whether or not the image contains foreign substances with X and Y appropriate for TENSORFLOW KERAS learning.

The learning unit (185) trains a deep learning model that uses the processed image as input data and the presence or absence of foreign substances as output data.

At this time, the learning unit (185) classifies the learning information data set into training, validation, and test data sets. Preferably, the ratio of training: validation: test is randomly classified to be 0.65: 0.2: 0.15. RandomSplitter can be used for random classification.

At this time, train_test_split can be used when the training dataset and test dataset are adjusted, and validation_pct can be used when the validation dataset is adjusted.

The learning unit (185) can learn the deep learning model by compiling layers of the deep learning model after augmenting the training data set and the validation data set. Preferably, the learning unit (185) can expand the diversity of the learning data and prevent overfitting by using a data augmentation technique when augmenting the data. Data augmentation can include image enlargement, cropping, rotation, random flip, random resizing, and brightness adjustment.

The learning unit (185) can transfer learn a deep learning model with the network structure of Resnet and Mobilenet using the Pytorch-based Fastai library. As a result, the deep learning model can be configured and compiled into an advanced layer.

The learning unit (185) can evaluate the model using a test data set, and save the deep learning model in pkl format and the evaluation values (accuracy, precision, recall, f-score, support) in csv format.

The deep learning model learned by the learning unit (185) can be transmitted to the PC (17) when requested by the PC (17).

FIG. 6 is a flowchart of a method for learning a deep learning model according to one embodiment of the present disclosure.

Referring to Fig. 6, when an image captured by the MWIR camera (150) is input by the input unit (180), it is processed by the image processing unit (183) provided in the server (18) (S600).

The processed image and the corresponding presence or absence of foreign substances are created and stored as a learning information dataset (S610).

Using the saved learning information data set, the learning unit (185) trains a deep learning model (S620). At this time, the deep learning model can use the processed image as input data and the presence or absence of foreign substances as output data. The deep learning model is trained on the server (18) and downloaded to the PC (17).

An image captured and processed by a camera device (15) on a PC (17) When transmitted, the PC (17) inputs the received processed image into the stored deep learning model to output whether or not it contains a predicted foreign substance (S630).

The PC (17) can provide the user with the predicted presence of foreign substances through the output unit (171), and the foreign substance inspection system (1) can control all or part of the components of the foreign substance inspection system (1) using the predicted presence of foreign substances.

Fig. 7 is a comparison between a foreign substance inspection system according to one embodiment of the present disclosure and a conventional inspection system. Meanwhile, Fig. 7 photographs an inspection target product packaged in a packaging material formed of a material including NY, PE, and LDPE.

FIG. 7(a) shows a comparison of the results of photographing the same inspection target product using a camera device (15) according to one embodiment of the present disclosure, that is, a MWIR camera (150), and the results of photographing the same product using a conventional camera device, specifically, a SWIR and X-ray camera.

Referring to Fig. 7(a), it can be confirmed that the MWIR camera (150) clearly distinguishes between food and low-density foreign substances in the product to be inspected. On the other hand, it can be seen that the results of the photographing using the SWIR camera barely penetrate the packaging material. Furthermore, it can be seen that in some areas where it penetrates, the image that could be considered low-density foreign substances is not distinguished. In addition, it can be seen that the results of the photographing using the X-ray camera do not show any part that could be judged as low-density foreign substances.

Through these results, it can be seen that the foreign substance inspection system (1) according to one embodiment of the present disclosure has a superior ability to detect low-density foreign substances compared to conventional methods.

FIG. 7(b) shows a comparison of the results of photographing the same inspection target product using a camera device (15) according to one embodiment of the present disclosure, that is, a MWIR camera (150), and the results of photographing the same product using a conventional camera device, specifically, a LWIR and X-ray camera.

Referring to Fig. 7(b), it can be seen that the MWIR camera (150) can capture detailed and clear images of internal materials through the packaging material. On the other hand, it can be seen that the images taken using the LWIR camera barely penetrate the packaging material. In addition, the images taken using the X-ray camera show that the packaging material penetration ability is excellent, but it is difficult to distinguish between low-density foreign substances and food.

Through these results, it can be seen that the foreign substance inspection system (1) according to one embodiment of the present disclosure has excellent packaging material penetration ability and, at the same time, excellent low-density foreign substance detection ability.

The above description is merely an illustrative description of the technical idea of the present embodiment, and those with ordinary skill in the art to which the present embodiment belongs may make various modifications and variations without departing from the essential characteristics of the present embodiment. Therefore, the present embodiments are not intended to limit the technical idea of the present embodiment, but to explain it, and the scope of the technical idea of the present embodiment is not limited by these embodiments. The protection scope of the present embodiment should be interpreted by the following claims, and all technical ideas within a scope equivalent thereto should be interpreted as being included in the scope of the rights of the present embodiment.

[Explanation of symbols]

1: Foreign substance inspection system

10: PLC

11: Defective sensor

12: Rejector

13: Conveyor belt

14: Warning lights

15: Camera Device

16: Product detection sensor

17: PC

18: Server

19: Database

Claims (11)

A conveyor belt configured to transport the product to be inspected; A product detection sensor that detects the introduction of a product to be inspected onto the conveyor belt; A PLC that generates a camera device trigger signal and transmits it to the camera device when the introduction of the product to be inspected is detected by the above product detection sensor; The camera device configured to photograph the product to be inspected and process the photographed image; and A PC configured to receive a deep learning model learned by a server and output whether or not a predicted foreign substance is included by inputting an image processed by the camera device into the deep learning model; The above camera device, A MWIR camera that photographs the product to be inspected using a wavelength of 3 ㎛ to 5 ㎛; and An image processing unit configured to process a raw image captured by the MWIR camera; Foreign substance inspection system. In the first paragraph, The above server, A thermal image input module configured to receive raw images captured by a MWIR camera; A foreign substance inclusion input module configured to receive input on whether or not a product to be inspected contains foreign substances; An image processing unit configured to process the input raw image and generate a processed image; A learning information storage unit that creates and stores a learning information dataset using the processed image and whether or not the foreign matter is included; and A learning unit configured to learn a deep learning model using the processed image as input data and the presence or absence of foreign substances as output data; including; Foreign substance inspection system. In the second paragraph, The image processing unit generates a processed image by binarizing the raw image and converting the binarized image to a preset size. The above PC, when receiving an image processed by the image processing unit, outputs whether there is a predicted foreign substance of 0 or 1. Foreign substance inspection system. In the third paragraph, If the predicted foreign substance inclusion is 0, The above PC displays the raw image or the processed image through the output section, The raw image or processed image of the product to be inspected above, the metadata of the raw image and the 0 value are stored in a database as a dataset. Foreign substance inspection system. In the third paragraph, If the predicted foreign substance inclusion is 1, The above PC sends 1 to the above PLC, The above PLC waits for a predetermined amount of time, then generates a lighting signal and transmits the lighting signal to the warning light. The above PLC sets counter 1 in the memory, generates a rejector operation signal, and transmits the rejector operation signal to the rejector. When the above rejector receives the above rejector operation signal, the product to be inspected corresponding to the presence or absence of the predicted foreign matter is moved to the defective product loading space. Foreign substance inspection system. In paragraph 5, The above defective product loading space is equipped with a defective product sensor configured to detect the introduction of the moved inspection target product, When the PLC receives a detection signal from the defective product sensor within a preset time from the time when counter 1 is set in the memory, the PLC clears the memory, If the PLC does not receive a detection signal from the defective product sensor within a preset time from the time at which the memory counter 1 is set, the PLC generates a conveyor belt stop signal and transmits the conveyor belt stop signal to the conveyor belt. Foreign substance inspection system. (a1) A step in which a deep learning model is trained by a server using the processed image of the product to be inspected as input data and the presence or absence of foreign substances as output data; (a2) a step in which the above deep learning model is transmitted to a PC; (b1) A step in which the introduction of a product to be inspected is detected by a product detection sensor, and the product detection sensor generates a product introduction signal and transmits it to a PLC; (b2) a step in which the PLC receives the product inlet signal, waits for a predetermined time, generates a trigger signal, and transmits the trigger signal to the camera device; (b3) a step of the camera device capturing a raw image of the product to be inspected using a MWIR camera, and the camera device processing the image using an image processing unit; (b4) a step of transmitting the processed image to the PC; and (c) a step of outputting whether the processed image contains a predicted foreign substance when the processed image is input to the deep learning model; including, The above MWIR camera photographs the product to be inspected using a wavelength of 3 ㎛ to 5 ㎛. Foreign substance inspection method. In Article 7, The above step (a1) is: (a10) A step in which the thermal image input module of the above server receives a raw image captured using a MWIR camera; (a11) A step in which the input module for determining whether the server contains foreign substances receives input of whether the raw image captured in the step (a10) contains foreign substances; (a12) a step of generating a processed image by processing the raw image input in step (a10) by the image processing unit of the server; and (a13) a step for the learning unit of the server to train a deep learning model using the processed image generated in step (a12) as input data and the presence or absence of foreign substances input in step (a11) as output data; Foreign substance inspection method. In Article 8, Step (c) above, (c10) A step in which the predicted presence or absence of foreign substances is output as 0; (c11) a step of the PC displaying the raw image or the processed image through the output unit; and (c12) a step of transmitting and storing the raw image or processed image of the product to be inspected, the metadata of the raw image, and the 0 value as a data set in a database by the PC; Foreign substance inspection method. In Article 8, Step (c) above, (c20) A step in which the presence or absence of the predicted foreign substance is output as 1; (c21) A step in which the PC transmits 1 to the PLC; (c22) The PLC waits for a predetermined period of time, then generates a lighting signal and transmits the lighting signal to the warning light; (c23) The PLC sets counter 1 in the memory and generates a rejector operation signal to generate the rejector operation signal; and (c24) further comprising a step of moving the inspection target product corresponding to the presence or absence of the predicted foreign substance in the defective product loading space when the rejector receives the rejector operation signal; Foreign substance inspection method. In Article 10, The above defective product loading space is equipped with a defective product sensor configured to detect the introduction of the moved inspection target product, Step (c) above, (c25) If the PLC receives a detection signal from the defective product sensor within a preset time from the ten points where the counter 1 is set in the memory, the PLC clears the memory; and (c26) If the PLC does not receive a detection signal from the defective product sensor within a preset time from the time when the counter 1 is set in the memory, the PLC generates a conveyor belt stop signal and transmits the conveyor belt stop signal to the conveyor belt; further comprising; Foreign substance inspection method.

Images