JP7185958B1 - PALLET DETECTION DEVICE AND PALLET DETECTION METHOD - Google Patents

Abstract

A pallet detection device capable of quickly and accurately ascertaining the number of pallets placed on a loading platform by a simpler method.
A pallet detection device 10 of the present invention includes an imaging unit 11 for acquiring first image information including an image of a pallet loaded on a loading platform, and geometrically transforming the shape of the first image information. an image processing unit 13 for acquiring second image information having a predetermined shape and determining whether the second image information is an image corresponding to a predetermined angle of view; and a palette detection unit 14 for detecting the image of the palette from the second image information by comparing the image information of the palette with the image information of the palette registered in advance. The image aligned with the bottom pallet placed on the floor of the loading platform detected by the pallet is detected as the image of the pallet.
[Selection drawing] Fig. 2

Description

The present invention relates to a pallet detection technique for detecting a pallet placed on a loading platform.

As a technology for managing packages in a warehouse, a method has been proposed for managing the installation locations of packages and pallets in the warehouse by reading identification codes assigned to the packages and pallets. Recently, a technology that reduces the burden of inspection and inventory work of packages in the warehouse by acquiring image information of packages and pallets in the warehouse using cameras mounted on vehicles and drones that move in the warehouse. is proposed.

For example, Patent Literatures 1 and 2 propose a pallet detection system using a robot or drone equipped with a camera that captures images of packages in a warehouse. It is possible to obtain information on the type and number of packages by reading barcodes installed on packages and shelves with cameras mounted on robots and drones.

JP 2019-527172 A JP 2019-167177 A

Here, with the above-described technology for reading barcodes installed on packages and pallets, it is necessary to acquire images of packages and pallets from a relatively short distance and from the front direction. When inspecting a package or pallet that has been packed, it may not be possible to detect the package or pallet accurately. At the time of loading and unloading loads placed on truck beds, etc., there is a technology that can accurately ascertain the number of loads and pallets in a simpler, quicker and more accurate manner than for loads placed in warehouses. Desired.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a pallet detecting device capable of quickly and accurately ascertaining the number of pallets placed on a loading platform in a simpler manner. aim.

In order to solve the above-described problems, the pallet detection device of the present invention includes an imaging unit that acquires first image information including an image of a pallet loaded on a loading platform, and a shape of the first image information. an image processing unit that acquires second image information having a predetermined shape by performing geometric transformation and determines whether the second image information is an image corresponding to a predetermined angle of view; When the image information is an image corresponding to a predetermined angle of view, the palette image is extracted from the second image information by comparing the second image information with the pre-registered palette image information. a pallet detection unit for detecting, the pallet detection unit detects an image aligned with the bottom pallet arranged on the floor surface of the loading platform detected in the second image information as the image of the pallet. do.

Further, the imaging section may acquire the first image information by performing at least one of zoom-in and zoom-out based on the result of the determination by the image processing section.

Further, the pallet detection unit detects a line profile of the second image information, a pattern matching result between the second image information and an image of pallet holes prepared in advance, and a palette in the frequency analysis result of the second image information. The lowest pallet placed on the floor surface of the loading platform may be detected based on any of the hole pattern extraction results.

In order to solve the above-described problems, the pallet detection method of the present invention provides an imaging unit that acquires an image of a pallet, an image processing unit that processes the image acquired by the imaging unit, and a A pallet detection method executed in a pallet detection device having a pallet detection unit for detecting an image of the pallet from an image received, wherein the imaging unit receives first image information including an image of the pallet loaded on a loading platform. and the image processing unit obtains second image information having a predetermined shape by geometrically transforming the shape of the first image information, and obtains the second image information having a predetermined shape. an image processing step of determining whether the image information is an image corresponding to a predetermined angle of view; an image detection step of detecting the image of the palette from the second image information by comparing the image information of the palette with the image information of the palette registered in advance; An image aligned with the bottom pallet arranged on the floor surface of the loading platform detected in the image information of 2 is detected as the image of the pallet.

The image acquisition step may acquire the first image information by performing at least one of zooming in and zooming out based on the result of the determination in the image processing step.

Further, the pallet detection unit detects a line profile of the second image information, a pattern matching result between the second image information and an image of pallet holes prepared in advance, and a palette in the frequency analysis result of the second image information. The lowest pallet placed on the floor surface of the loading platform may be detected based on any of the hole pattern extraction results.

ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to provide the pallet detection apparatus which can grasp|ascertain the number of pallets installed in the loading platform rapidly and correctly by a simpler method.

FIG. 1 shows an example of a system configuration including a pallet detection device according to an embodiment of the present invention. FIG. 2 is an example of functional blocks of the pallet detection device according to the embodiment of the present invention. FIG. 3 is a diagram for explaining a palette detection method according to the embodiment of the present invention. FIG. 4 is a diagram for explaining a method of detecting the bottom pallet according to the embodiment of the present invention. FIG. 5 is a diagram for explaining the palette detection result in the embodiment of the present invention. FIG. 6 is an example of the operation flow of the pallet detection method according to the embodiment of the present invention.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings. The present invention can be implemented in various embodiments, and is not limited to the embodiments described below.

<Configuration of pallet detection system>
FIG. 1 is an example of the configuration of a pallet detection system according to an embodiment of the present invention. The pallet detection system of FIG. 1 acquires an image of a pallet 3 in a state where a load 2 is placed on the bed of a carrier vehicle such as a truck 1, and uses the acquired image to determine the type of the pallet 3 on the bed. It comprises a pallet detection device 10 for detecting the number of pallets, and a pallet management server 20 for managing the type and number of pallets detected by the pallet detection device 10 .

The pallet detection system of FIG. 1 compares the image information acquired by the imaging unit mounted on the pallet detection device 10 with the image information of the pre-registered pallet, so that the cargo 2 loaded on the bed of the truck 1 is placed. It is configured to detect the pallet 3 in the folded state. The image in which the palette 3 is detected and the detection result of the palette 3 are sent to the palette management server 20 as required.

When the cargo 2 placed on the loading platform of the truck 1 or the like is carried out or carried in, it is necessary to detect the number of pallets 3 installed on the loading platform of the truck 1 or the like in order to check whether the cargo to be transported is loaded. There is For this reason, there are time and place restrictions compared to checking the packages installed in the warehouse, so there is a demand for a simpler method to quickly and accurately ascertain the number of packages and pallets. be.

In the configuration example of FIG. 1, pallets 3 on which a plurality of loads 2 are placed are arranged in a state of being stacked in a plurality of stages on the loading platform of the truck 1 . Since the type of pallet 3 on the loading platform of the truck 1 is not limited to one type, it is necessary to grasp the types and numbers of multiple types of pallets by acquiring an image of the entire load.

As shown in the rear view of FIG. 1, the cargo bed of the truck 1 in the configuration example of FIG. 1 has a structure divided into left and right sections. It has the depth of the sheet. In such a loading platform, the number of pallets in the entire loading platform can be grasped by detecting pallets using images acquired from the outside of each of the left and right loading platforms.

The imaging unit of the pallet detection device 10 is an imaging element for acquiring image information of the entire loading platform including the image of the pallet. As the imaging unit, a general-purpose RGB camera for detecting the color and shape of the pallet from the image information of the entire loading platform can be used. For example, a digital camera equipped with an RGB camera, a smart phone, or the like can be used as the palette detection device 10 having an imaging unit.

The pallet detection device 10 acquires image information of the entire loading platform including the image of the pallet 3, and compares it with prestored image information of the pallet to detect the image of the pallet from the image information of the entire loading platform. The image information of the palette that is stored in advance includes information for expressing the characteristics of the palette such as the color and shape of the palette.

As the palette image information, captured images may be stored for each type of palette, or image information learned using image information of a large number of palettes may be used. From the image information of the entire loading platform including the image information of the pallet 3, the type and number of pallets on the loading platform can be grasped by detecting the information for representing the characteristics of the pallet such as the color and shape of the pallet.

When loading or unloading a package installed on the loading platform of the truck 1 or the like, the photographing position of the photographer who takes the image of the loading platform is not always the same. The distance and angle between the carriers are not constant depending on the photographer. Therefore, the image information of the pallet stored in advance cannot be detected in the acquired loading platform image, and the detection accuracy of the pallet image may be lowered. The pallet detection device of the present embodiment is configured to improve the detection accuracy of the pallet image by performing image processing such as geometric transformation on the acquired bed image.

Specifically, if the distance or angle between the digital camera or smartphone and the cargo bed differs depending on the person who takes the image of the cargo bed, the shape of the cargo bed image will be distorted. When the distortion of the image is corrected so that the photographed image has a predetermined shape by geometrically transforming the shape of the information), and the image having the predetermined shape with the corrected distortion corresponds to the predetermined angle of view (2) detects image information of a palette using an image (second image information) having a predetermined shape whose distortion has been corrected.

On the other hand, if the distortion-corrected image having the predetermined shape does not correspond to the predetermined angle of view, the imaging unit is zoomed in or out as necessary, and the information of the cargo bed image is acquired again. Then, image processing such as geometric transformation is performed on the acquired loading platform image to acquire an image having a predetermined shape in which distortion is corrected.

In this way, the distortion of the image is corrected so that the photographed image has a predetermined shape, and the pallet image is compared with the image information of the pallet using the pallet image corresponding to the predetermined angle of view. It is possible to improve the detection accuracy of the image information of the palette when detecting the image information of the palette.

As a method for geometrically transforming the loading platform image, for example, trapezoidal correction for correcting a trapezoidal image into a rectangular image can be considered, but the method is not limited to trapezoidal correction. It is not limited to trapezoidal correction, and image conversion may be performed so that a rectangular image having an arbitrary shape becomes an image corresponding to a predetermined angle of view.

When an image of a required size is obtained by geometric transformation, it is not always necessary to zoom in or zoom out of the imaging section. The image corresponding to the predetermined angle of view for detecting the image information of the palette may be obtained by zooming in or out of the imaging section so that the area is approximately the same as the image size at the predetermined angle of view, or by zooming out the image. It is conceivable to zoom in or out the imaging unit so that the size falls within a predetermined range at a predetermined angle of view. The imaging unit may be zoomed in or out so that the size of the image corresponds to the image information of the cargo bed learned in advance.

<Configuration of Pallet Detection Device>
The configuration of the pallet detection device 10 will be described with reference to FIG. FIG. 2 is an example of functional blocks of the pallet detection device according to the embodiment of the present invention.

The pallet detection device 10 includes an imaging unit 11 for acquiring image information of the loading platform including image information of the pallet, a central processing unit 12 for performing image processing and pallet detection on the image acquired by the imaging unit 11, and a pallet management server. 20, a storage unit 15 for storing various information, an input unit 16 for the photographer to input information, and a display unit for displaying the detection status of the palette image. 17.

The central processing unit 12 functions as an image processing unit 13 that processes the image acquired by the imaging unit 11 and a palette detection unit 14 that detects a palette. The pallet detection device 10 can be composed of a computer having a CPU, a memory, and an interface for connecting with an external device.

The storage unit 15 can store pallet image information and pallet detection results for comparison with the acquired loading platform image information. The image information of the palette includes information for expressing the characteristics of the palette such as the color and shape of the palette. As the palette image information, image data obtained by learning image data of a plurality of palettes may be used.

<Pallet detection method>
3 and 4 are diagrams for explaining the palette detection method according to the embodiment of the present invention. FIG. 3 shows an example of image processing, such as geometric transformation, on the carrier image.

In the example of FIG. 3, geometric transformation is performed so that the reference position (a, b, c, d) on the loading platform becomes a rectangular image (A, B, C, D), and the image pickup unit 11 zooms in. Geometric transformation is performed again on the obtained cargo bed image. As a result of performing the geometric transformation again, an image is obtained in which the area of the image for detecting the package image is within a predetermined range at a predetermined angle of view.

In FIG. 3, the image obtained by geometrically transforming the image zoomed in by the imaging unit 11 is within a predetermined range at a predetermined angle of view. It is an image of By using this image and comparing it with the image information of the palette 3, the type and number of the palette 3 can be specified. The image information acquired by the imaging unit 11 is geometrically transformed, and the image is zoomed in or out as necessary, and is compared with the image information of the palettes stored in advance for each type of the palette 3. , it is possible to identify the type of pallets 3 on the loading platform and grasp the number of pallets 3 for each type of pallet 3 . Zoom-in or zoom-out in the imaging unit 11 may be performed by electronic zooming by the imaging unit 11, optical zooming by an optical lens of the imaging unit 11, or the like.

In the method of detecting the type and number of palettes using pre-saved palette image information, there is a method of determining that the image is a palette image when the difference from the palette image information is less than a predetermined threshold. Conceivable. In this method, an image whose difference from palette image information is close to the threshold may be erroneously detected as a palette image.

As shown in FIG. 1, when a plurality of pallets on which cargo is placed are loaded on the loading platform of a transport vehicle such as a truck, in order to stabilize the loading state of the cargo, a pallet is arranged on the floor of the loading platform. A loading method is adopted in which pallets are loaded so as to align with the bottom pallet.

In the pallet detection method of the present embodiment, this loading method is used to improve the detection accuracy of the pallet image by using the position information of the lowermost pallet on the truck bed. Specifically, an image aligned with the image of the pallet at the bottom of the loading platform is detected as an image of the pallet, and an image aligned with the image of the pallet at the bottom of the loading platform is excluded as an erroneously detected palette.

FIG. 4 is a diagram for explaining a method of detecting the bottom pallet according to the embodiment of the present invention. In FIG. 4, the pallet at the bottom of the cargo bed is detected using the line profile of the image information of the cargo. In the image information geometrically transformed in FIG. 3, the image information is searched in the horizontal direction to acquire the line profile of the image information. The detected position is detected as the position of the lowest pallet of the loading platform.

The method of detecting the lowermost pallet is not limited to the method of detecting the line profile corresponding to the hole in the pallet described above. Other methods such as a method of pattern matching between the image information of the package and the image of the pallet holes prepared in advance, and a method of extracting a pattern with the same cycle as the pallet holes by frequency analysis of the data of the image information of the package. may be used to detect the bottom pallet.

As a result of pallet image detection, the image that is aligned with the image of the pallet on the bottom of the loading platform is detected as the image of the pallet, and the image that is not aligned with the image of the pallet on the bottom of the loading platform is detected as an incorrectly detected pallet. exclude. As a result, the detection accuracy of the palette image can be improved by eliminating the case where an image having a difference from the image information of the palette near the threshold value is erroneously detected as the palette image.

<Pallet detection result>
FIG. 4 shows an example of a palette detection result when there are three palette types (colors). The image information acquired by the imaging unit 11 is geometrically transformed, and the image is zoomed in or out as necessary, and is compared with the image information of the palettes stored in advance for each type of the palette 3. , it is possible to identify the type of pallets 3 on the loading platform and grasp the number of pallets 3 for each type of pallet 3 . In the example of FIG. 4, there are three types of pallets 3, and the number of sheets of each pallet is "6", "3", and "5".

As explained in FIG. 4, images that are not aligned with the image of the pallet at the bottom of the loading platform are excluded as erroneously detected pallets. As a result, the detection accuracy of the palette image can be improved by eliminating the case where an image having a difference from the image information of the palette near the threshold value is erroneously detected as the palette image.

In FIGS. 4 and 5, the pallet image in the image information of the cargo is detected, and from the detection result of the pallet image in the image information of the cargo, the image that is not aligned with the image of the pallet at the bottom of the cargo bed is regarded as an incorrectly detected pallet. Although excluded, the pallet image detection process may be performed on the assumption that the image of the pallet is aligned with the image of the pallet at the bottom of the loading platform in the image information of the luggage.

According to the present embodiment, even if the distance or angle between the digital camera or smartphone and the cargo bed is not constant by the photographer who takes the image of the cargo bed of the truck 1 or the like, geometric transformation or the like can be applied to the acquired image. By performing image processing, it is possible to improve the detection accuracy of the pallet image of the load placed on the loading platform of the truck 1 or the like.

<Operation flow of pallet detection device>
A pallet detection operation in the pallet detection device 10 will be described with reference to FIG. FIG. 5 is an example of the operation flow of the pallet detection method according to the embodiment of the present invention.

The imaging unit of the pallet detection device acquires the image of the loading platform including the image of the pallet (S1-1: image acquisition step). The image processing unit of the pallet detection device detects a predetermined reference position from the image of the loading platform (S1-2), and geometrically transforms the loading platform image so that the shape of the image formed from the reference positions becomes a predetermined shape. (S1-3: image processing step).

The image processing unit of the pallet detection device further determines whether the area of the cargo bed image after geometric transformation corresponds to a predetermined angle of view (S1-4, S1-5), and (S1-5: YES), the image of the pallet is detected from the image of the pallet by comparing the image information of the pallet registered in advance with the image of the pallet after the geometric transformation (S1-5: YES). 7: image detection step).

If the cargo bed image after geometric transformation does not correspond to the predetermined angle of view (S1-5: NO), the imaging unit of the pallet detection device zooms in or out to acquire the image information of the cargo bed again. Then, it is determined whether or not the cargo bed image after the geometric transformation of the image information is an image corresponding to a predetermined angle of view.

As a result of zooming in or out, if the image after geometric transformation corresponds to a predetermined angle of view (S1-5: YES), the cargo bed image after geometric transformation and the pre-registered pallet image are combined. By comparison, the image of the pallet is detected from the loading platform image (S1-7).

In the operation flow of FIG. 6, from the detection result of the pallet image in the image information of the luggage, the image that is not aligned with the image of the lowermost pallet arranged on the floor of the loading platform is excluded as an erroneously detected pallet (S1-8 ). As a result, the detection accuracy of the palette image can be improved by eliminating the case where an image having a difference from the image information of the palette near the threshold value is erroneously detected as the palette image.

The image in which the palette is detected and the palette detection result are sent to the palette management server as required. The detection results are verified by verifying the images in which the pallets are detected and the pallet detection results. If the detection results are incorrect, the corrected detection results are used to learn the pallet images to improve the detection accuracy. You may make it improve.

For example, when the user refers to the palette detected image and the palette detection result, it may detect an object that is not a palette as a palette (false detection), or conversely, when an image that should be detected as a palette is detected. If it is recognized that there was a case where it was not detected (detection omission), the falsely detected image is treated as an image other than the palette image, and the detection omission image is treated as the palette image, thereby improving detection accuracy. You may try to do so.

As described above, according to the present embodiment, it is possible to provide a pallet detection device capable of quickly and accurately ascertaining the number of pallets placed on the loading platform by a simpler method.

DESCRIPTION OF SYMBOLS 1... Truck 2... Baggage 3... Pallet 10... Pallet detection apparatus 11... Imaging part 12... Central processing part 13... Image processing part 14... Pallet detection part 15... Storage part 16... Input part , 17 ... display unit, 20 ... palette management server.

Claims (6)

an imaging unit that acquires first image information including an image of the pallet loaded on the loading platform;
Obtaining second image information having a predetermined shape by geometrically transforming the shape of the first image information, and determining whether the second image information is an image corresponding to a predetermined angle of view. an image processing unit;
When the second image information is an image corresponding to a predetermined angle of view, by comparing the second image information with the image information of the pre-registered palette, the and a palette detection unit for detecting an image of the palette,
The pallet detection unit detects, as the image of the pallet, an image in which the lowermost pallet arranged on the floor surface of the cargo bed detected in the second image information is aligned. The pallet detection device according to claim 1, wherein the imaging section acquires the first image information by performing at least one of zoom-in and zoom-out based on the result of the determination by the image processing section. The pallet detection unit detects a line profile of the second image information, a pattern matching result between the second image information and an image of pallet holes prepared in advance, and a pallet hole in the frequency analysis result of the second image information. 3. The pallet detecting device according to claim 1, wherein the lowest pallet placed on the floor surface of the loading platform is detected based on one of the pattern extraction results. A pallet detection device comprising an imaging unit for acquiring an image of a pallet, an image processing unit for image processing the image acquired by the imaging unit, and a pallet detection unit for detecting the pallet image from the image processed by the image processing unit. A palette detection method performed in
an image acquisition step in which the imaging unit acquires first image information including an image of a pallet loaded on a loading platform;
The image processing unit acquires second image information having a predetermined shape by geometrically transforming the shape of the first image information, and the second image information is obtained at a predetermined angle of view. an image processing step of determining whether the image is a corresponding image;
When the second image information is an image corresponding to a predetermined angle of view, the palette detection unit compares the second image information with image information of the palette registered in advance to detect the second image. an image detection step of detecting the image of the palette from the image information of 2,
The pallet detection method, wherein the pallet detection unit detects, as the image of the pallet, an image aligned with the lowermost pallet arranged on the floor surface of the loading platform detected in the second image information. 5. The palette detection method according to claim 4, wherein said image acquisition step acquires said first image information by performing at least one of zooming in and zooming out based on the result of said determination in said image processing step. The pallet detection unit detects a line profile of the second image information, a pattern matching result between the second image information and an image of pallet holes prepared in advance, and a pallet hole in the frequency analysis result of the second image information. 6. The pallet detection method according to claim 4 or 5, wherein the bottom pallet placed on the floor surface of the loading platform is detected based on any of the pattern extraction results.

Images