JP2022111040A - Optical information reader - Google Patents

Abstract

Kind Code: A1 A neural network structure specialized for restoring an image containing codes is used to improve processing speed while restoring images containing codes with a wide range of pixel resolutions.
A processor inputs a read image enlarged or reduced so that the pixel resolution of modules constituting the code is within a specific range, to a neural network configured with a structure and parameters stored in a storage unit. attempts to restore the read image, and decodes the restored read image.
[Selection drawing] Fig. 12

Description

The present invention relates to an optical information reader that optically reads information.

BACKGROUND ART In recent years, so-called traceability, which enables, for example, traceability of a distribution route of an article from a manufacturing stage to a consumption stage or a disposal stage, has been emphasized, and code readers for the purpose of this traceability have become popular. Code readers are also used in various fields other than traceability.

Generally, a code reader photographs a code such as a bar code or a two-dimensional code attached to a workpiece with a camera, extracts the code contained in the obtained image by image processing, binarizes it, and decodes it. It is configured to be able to read information, and is also called an optical information reader because it is a device for optically reading information.

As disclosed in Patent Document 1, for example, this type of optical information reading device learns a model structure showing the relationship between a code image acquired by a visual sensor of a robot and an ideal code image. There is known a machine equipped with a machine learning device that Patent Document 1 describes restoring an image suitable for reading by applying learning results from a machine learning device to an image of a code acquired by a visual sensor during operation.

Japanese Patent No. 6683666

By the way, in order to fully capture the features of the code and restore the image to be suitable for reading, it is necessary to extract feature quantities within a range that covers a certain number of modules. For example, if one feature value obtained from the neural network is a value extracted from a range of 6×6 modules, and one module is an image captured with 20 pixels, it is aggregated from a range of 120×120 pixels. A neural network must be designed to calculate feature values that

In other words, the wider the pixel range to be covered, the deeper the layer of the neural network must be. For example, to aggregate the feature values from the 120×120 pixel range described above, six convolution layers are required. Become.

However, since the code is composed of white modules and black modules arranged randomly, the relationship between pixel values over a wide range is weak. The effect is hardly improved.

The present invention has been made in view of this point, and its object is to improve the processing speed as a neural network structure specialized for inpainting images containing codes, including codes with a wide range of pixel resolutions. To enable restoration of an image.

In order to achieve the above object, one aspect of the present disclosure can presuppose an optical information reading device that reads a code attached to a work. An optical information reader includes a camera that captures a code and generates a read image, a plurality of defective images having portions inappropriate for reading the code, and a plurality of ideal images corresponding to the plurality of defective images. Among them, a storage unit that stores the structure and parameters of a neural network generated in advance by machine learning the defective image and the ideal image in which the pixel resolution of the modules constituting the code is within a specific range, and the camera A neural network configured with the structure and parameters stored in the storage unit for the generated read image, which is enlarged or reduced so that the pixel resolution of the modules constituting the code in the generated read image is within the specific range. and a processor that attempts to restore the read image according to the structure and parameters stored in the storage unit by inputting to and performs decoding processing on the restored read image.

According to this configuration, when generating the structure and parameters of the neural network, machine learning is performed on a defective image and an ideal image in which the pixel resolution of the modules constituting the code is within a specific range. The specific range is a range that includes pixel resolutions with a high restoration effect and excludes pixel resolutions that hardly improve the restoration effect, and can be set on the premise of restoration of an image containing a code composed of multiple modules. can. As a result, the processing speed can be improved as a neural network structure specialized for restoration of images containing codes.

In addition, if the pixel resolution of the modules that compose the code in the read image generated by the camera is outside the specific range, the read image is enlarged or reduced so that it falls within the specific range, and then sent to the neural network. input, allowing restoration of images containing codes with a wide range of pixel resolutions.

The arrangement of fixed module patterns such as the finder pattern included in the code and the rectangular shape of the entire code can be wide-range features, but if the module and the background are only roughly separated, the wide-range feature is used. It is also possible to recover satisfactorily using narrow-range features alone, without

In another aspect of the present disclosure, the structure and parameters of the neural network are generated in advance by machine-learning the defective image and the ideal image in which the number of pixels of the modules constituting the code is within a specific range, The processor can input to the neural network the read image generated by the camera, which is enlarged or reduced so that the number of pixels of the modules constituting the code in the read image is within the specific range. .

According to this configuration, by generating a neural network by limiting the number of pixels within a specific range, it is possible to obtain a neural network structure that is more specialized for restoration of images containing codes. The specific range of the number of pixels can be, for example, 4 PPC or more and 6 PPC or less, but it may be less than 4 PPC or more than 6 PPC. PPC is a unit that indicates how many pixels (picture elements) are in one module.

In addition, for example, by limiting the specific range to 4 PPC or more and 6 PPC or less for machine learning, it is possible to eliminate unnecessary scale fluctuations from the image used for learning, and to optimize the neural network structure.

According to another aspect of the present disclosure, a filter processing unit that executes an image processing filter on the read image generated by the camera before the enlargement or reduction of the read image is further provided. Appropriate image processing can be performed prior to enlargement or reduction, resulting in better and more accurate restoration of the read image. Examples of image processing filters include noise removal filters.

In another aspect of the present disclosure, when the optical information reading device is set, a plurality of read images with different enlargement ratios or a plurality of read images with different reduction ratios are generated, and the generated plurality of read images are processed by the neural network. Input to the network and try to restore each read image, execute decoding processing for each restored read image to obtain the read margin, and enlarge the read image whose read margin is higher than the predetermined read margin Alternatively, the optical information reading device further includes a tuning execution unit that specifies the reduction ratio as the enlargement ratio or reduction ratio to be used during operation. The processor can enlarge or reduce the read image to the enlargement ratio or reduction ratio specified by the tuning execution unit when the optical information reading device is operated.

That is, when the specific range has a certain extent, it is conceivable that the reading margin will change even within the specific range if the enlargement ratio or the reduction ratio is changed. With this configuration, it is possible to specify the enlargement ratio or reduction ratio of the read image during operation so that the reading margin is higher than the predetermined value, so that the processing speed and reading accuracy are improved.

In another aspect of the present disclosure, the tuning execution unit sets the enlargement ratio or reduction ratio of the read image with the highest read margin among the read margins of the read images as the enlargement ratio or reduction ratio to be used during operation. Since it is specified, processing speed and reading accuracy are further improved.

In another aspect of the present disclosure, the storage unit associates and stores structures and parameters of a plurality of neural networks with different numbers of layers or different numbers of image processing filters and code conditions. A tuning execution unit that sets code conditions included in the read image generated by the camera, reads the structure and parameters of a neural network associated with the set code conditions from the storage unit, and specifies a neural network to be used during operation. is further provided by the optical information reader. The processor can attempt restoration of the read image with the neural network specified by the tuning execution unit during operation of the optical information reading device.

That is, the optimal number of layers of the neural network or the number of filters can be pre-specified according to the code conditions, especially the PPC, so that they can be associated. Then, if code conditions are specified when setting up the optical information reader, the structure and parameters of the neural network associated with the code conditions can also be specified. As a result, it is possible to attempt restoration with the optimum neural network structure and parameters for the code conditions, thereby improving reading accuracy.

In another aspect of the present disclosure, the tuning execution unit sets code conditions included in the read image generated by the camera, acquires the number of pixels of the module that is the set code conditions, and the number of pixels acquired is If it is within the specified range, the enlargement or reduction by the processor is not executed, so that the enlargement or reduction can be performed only when the number of pixels of the module is outside the specified range.

In another aspect of the present disclosure, the storage unit is configured to store a plurality of parameter sets with different enlargement ratios by the processor or a plurality of parameter sets with different reduction ratios by the processor. The processor can perform decoding processing by applying any one parameter set from among the plurality of parameter sets stored in the storage unit. As a result, for example, when the number of pixels in a code module changes, the optical information reader can be operated by applying the parameter of the enlargement rate or reduction rate corresponding to the number of pixels.

In another aspect of the present disclosure, the processor searches for a code in the enlarged or reduced read image based on information for specifying the code, and detects a region including the searched code in which the code exists. By extracting a chord candidate region with a high probability and inputting a partial image corresponding to the extracted chord candidate region to a neural network configured with the structure and parameters stored in the storage unit, the partial image is stored in the storage unit. An attempt is made to inpaint the partial image according to the structure and parameters obtained, and a decoding process is performed on the inpainted partial image.

With this configuration, the size of the image input to the neural network can be reduced. As a result, the computation load on the processor is lightened, and the processing speed is increased. In addition, since the partial image corresponds to an area where the code is highly likely to exist, it is possible to create an image containing the information required for reading, that is, the entire code. Since the image containing the entire code is input to the neural network, a decrease in reading accuracy is suppressed.

In another aspect of the present disclosure, an image restoration unit that inputs a read image to a neural network and performs an inference process to generate a restored image obtained by restoring the read image, and a first decoding process for the read image generated by the camera. and a second decoding processing unit for executing second decoding processing on the restored image generated by the image restoration unit. The second decoding processing section extracts a code candidate region from the read image and sends a trigger signal to the image restoration section to execute inference processing. When receiving the trigger signal sent from the second decoding processing unit, the image restoration unit inputs the partial image corresponding to the code candidate area to the neural network, and executes inference processing to generate a restored image by restoring the partial image. do. The second decoding processing unit can determine grid positions indicating each cell position of the code based on the restored image, and decode the restored image based on the determined grid positions. That is, the first decoding processing unit decodes a normal read image that is not subjected to inference processing, the image restoration unit generates a restored image by inference processing, and the restored image restored by inference processing By having the second decoding processing unit execute the decoding of , it is possible to construct a processing circuit suitable for each of them and to increase the processing speed. In addition, since the image restoration section only needs to restore the partial image corresponding to the code candidate area, the processing speed can be further increased.

In another aspect of the present disclosure, processing speed can be increased by executing inference processing by the image restoration unit and first decoding processing by the first decoding processing unit in parallel. In addition, since at least a part of the period during which the second decoding processing unit is executing the second decoding processing has a pause period during which the inference processing by the image restoration unit is not executed, the load on the image restoration unit can be reduced, Heat generation can be suppressed. In this case, the second decoding processing section may be configured with a plurality of cores.

In another aspect of the present disclosure, the first decoding processing section can execute the first decoding process in a shorter time than the time required for the second decoding process. Further, the setting unit can switch whether or not to execute the second decoding process, that is, whether or not to execute the restored image decoding process. Since the second decoding process is for decoding the restored image, it takes longer than the first decoding process for decoding the normal read image. On this premise, if it is set to execute the second decoding process, it is possible to set a decoding timeout time (decoding limit time) longer than the time required to execute the second decoding process. On the other hand, if it is set not to execute the second decoding process, a decode timeout period shorter than the time required to execute the second decoding process can be set. When it exceeds, it is possible to quickly shift to the decoding process of the next captured image.

The decoding processing unit according to another aspect of the present disclosure can execute the second decoding process on the restored image in parallel with the first decoding process on the read image generated by the camera. processing speed can be increased. When the setting unit has set not to execute the second decoding process, the decoding process part can allocate processing resources used for the second decoding process to the first decoding process. The processing resource is, for example, a memory usage area, a core constituting a multi-core CPU, or the like. When the second decoding process is set not to be executed, the memory area and the core can be used for the first decoding process, so that the first decoding process can be further speeded up.

A decoding processing unit according to another aspect of the present disclosure extracts a code candidate area in which a code is likely to exist from a read image. The image restoration unit inputs a partial image corresponding to the code candidate area to the neural network and generates a restored image by restoring the partial image. The decoding processing unit determines a grid position based on the restored image generated by the image restoration unit, and decodes the restored image based on the determined grid position. That is, since it often takes a long time to generate a repaired image, by extracting a code candidate region based on, for example, a feature value indicating code-likeness before generating a repaired image, Save time. Also, since the grid position can be determined with high accuracy based on the restored image, reading performance is improved.

In another aspect of the present disclosure, tuning processing is performed to determine the optimum imaging conditions and decoding conditions and to set the size of the code to be read. The decoding processing unit can determine the size of the extracted image including the code candidate area from the read image based on the size of the code set in the tuning process. By enlarging/reducing the extracted image to an image of a predetermined size and inputting it to the neural network, the PPC of the code image input to the neural network can be kept within a predetermined range. Restoration effect can be stably drawn out. In addition, although the size of the code to be read may vary, since the size of the code that is finally input to the neural network can be fixed, the balance between high speed processing and ease of decoding can be achieved. can keep.

A camera according to another aspect of the present disclosure receives light that passes through the first polarizing plate and is reflected from the code through the second polarizing plate. to generate a low-contrast read image. By using the polarizing plate, the specular reflection component of the workpiece is removed, so that a read image can be acquired with reduced influence of the specular reflection component. For example, a polarizing plate is suitable for an image with many regular reflection components, such as when an image of a metal workpiece is captured, but a decrease in the amount of light may darken the read image and lower the contrast. In such a case, reading performance is improved by converting the image into a high-contrast restored image through neural network inference processing.

In another aspect of the present disclosure, the tuning execution unit can set a plurality of imaging conditions and code conditions including a first imaging condition and code condition and a second imaging condition and code condition. The decode processing unit inputs the read image generated under the first imaging condition and the code condition to the neural network to generate the restored image, and if the decode processing of the restored image fails, the first imaging condition and the code condition are generated. A read image generated under second imaging conditions and code conditions different from the code conditions is input to the neural network to generate a restored image, and the restored image can be decoded.

As described above, a neural network generated in advance by machine-learning a defective image and an ideal image in which the pixel resolution of the modules composing the code is within a specific range is applied to the modules composing the code in the read image. By inputting a read image that has been enlarged or reduced so that the pixel resolution is within a specific range, the neural network can attempt to restore the read image. This makes it possible to attempt restoration of images containing codes with a wide range of pixel resolutions while improving processing speed as a neural network structure specialized for restoration of images containing codes.

It is a figure explaining the time of operation of a stationary optical information reader. 1 is a perspective view of a stationary optical information reader; FIG. 1 is a block diagram of an optical information reader; FIG. 1 is a perspective view of a handheld optical information reader; FIG. It is a figure explaining each part comprised by a processor. 1 is a conceptual diagram of a neural network; FIG. 4 is a flow chart showing an example of a basic procedure during learning of a neural network; A pair of a defective image and an ideal image is shown, where (A) is an example of a defective image and (B) is an example of an ideal image. 1 is a conceptual diagram of a convolutional neural network used for image conversion; FIG. 4 is a flow chart showing an example of a procedure of a tuning process performed when setting the optical information reading device; 7 is a flow chart showing an example of a decoding processing procedure before determination of a reduction ratio and an enlargement ratio; 7 is a flow chart showing an example of a decoding process procedure after determination of a reduction rate and an enlargement rate; (A) shows an example of a read image generated by a camera, and (B) shows an example of an image after execution of an image processing filter. (A) shows an example of an image after reduction processing, and (B) shows an example of an image after code region extraction. FIG. 10 is a diagram showing an example of restoration of a read image by a neural network; It is a figure which shows an example of the decoding processing procedure which concerns on other embodiment. 4 is a time chart of decoding processing by a first decoding processing unit and a second decoding processing unit; 4 is a time chart of decoding processing by a decoding processing unit; 4 is a flowchart showing an example of first decoding processing and second decoding processing; It is a figure which shows the image example of each process. FIG. 10 is a diagram for explaining a case in which a plurality of neural networks are used to handle black-and-white inversion of a code; FIG. 10 is a diagram for explaining a case in which one neural network is used to handle black/white inversion of a code; 4 is a graph showing the relationship between contrast and matching level; 7 is a flowchart illustrating an example of matching level calculation processing; FIG. 4 is a diagram showing an example of a user interface displayed on the display unit; It is a figure which shows the example using an imaging device with an AI chip.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail based on the drawings. It should be noted that the following description of preferred embodiments is essentially merely illustrative, and is not intended to limit the invention, its applications, or its uses.

(Stationary Optical Information Reader)
FIG. 1 is a diagram schematically showing operation of a stationary optical information reader 1 according to an embodiment of the present invention. In this example, a plurality of works W are conveyed in the direction of arrow Y in FIG. Such an optical information reader 1 is installed. The optical information reader 1 is a code reader configured to photograph a code attached to a work W, decode the code contained in the photographed image, and read information. In the example shown in FIG. 1, the optical information reader 1 is of a stationary type. When the stationary optical information reader 1 is operated, it is fixed to a bracket or the like (not shown) so that the optical information reader 1 does not move. The stationary optical information reader 1 may be used while being held by a robot (not shown). Alternatively, the code of the work W in a stationary state may be read by the optical information reader 1 . The operating time of the stationary optical information reading device 1 is the time when the code of the work W conveyed by the conveying belt conveyor B is sequentially read.

A code is attached to the outer surface of each work W. As shown in FIG. Codes include both barcodes and two-dimensional codes. Examples of two-dimensional codes include QR code (registered trademark), micro QR code, data matrix (Data code), Veri code, Aztec code, PDF417, and Maxi code. and so on. Two-dimensional codes include stack type and matrix type, but the present invention can be applied to any two-dimensional code. The code may be attached by printing or stamping it directly on the work W, or by attaching it to the work W after being printed on a label.

The optical information reader 1 is wired to a computer 100 and a programmable logic controller (PLC) 101 via signal lines 100a and 101a, respectively. A communication module may be incorporated in the PLC 101 to wirelessly connect the optical information reader 1 with the computer 100 and the PLC 101 . The PLC 101 is a control device for sequence-controlling the transport belt conveyor B and the optical information reader 1, and a general-purpose PLC can be used. As the computer 100, a general-purpose or dedicated computer, a portable terminal, or the like can be used.

During operation, the optical information reader 1 receives a read start trigger signal that defines the start timing of code reading from the PLC 101 via the signal line 101a. Then, the optical information reader 1 picks up and decodes the code based on this reading start trigger signal. After that, the decoded result is transmitted to the PLC 101 via the signal line 101a. As described above, when the optical information reader 1 is operated, the input of the reading start trigger signal and the output of the decoding result are performed between the optical information reader 1 and the external control device such as the PLC 101 via the signal line 101a. Repeatedly. The reading start trigger signal input and the decoding result output may be performed via the signal line 101a between the optical information reader 1 and the PLC 101 as described above, or other signals (not shown) may be input. You can go through the line. For example, a sensor for detecting the arrival of the work W may be directly connected to the optical information reader 1, and a reading start trigger signal may be input to the optical information reader 1 from the sensor.

As shown in FIG. 2, the optical information reader 1 includes a box-shaped housing 2, a polarizing filter attachment 3, an illumination section 4, a camera 5, a display section 6, a power connector 7, and a signal line connector. 8 are provided. Further, the housing 2 is provided with an indicator 9, an aimer light irradiation section (light irradiation section) 10, and operation buttons 11 and 12. The indicator 9, the aimer light irradiation section 10 and the operation buttons 11 and 12 are also provided. It is a component of the optical information reader 1 .

The housing 2 has a shape elongated in a predetermined direction, but the shape of the housing 2 is not limited to the illustrated shape. A polarizing filter attachment 3 is detachably attached to the front outer surface of the housing 2 . The housing 2 accommodates an illumination unit 4, a camera 5, an aimer light irradiation unit 10, a processor 20, a storage unit 30, a ROM 40, a RAM 41, and the like. The processor 20 , storage unit 30 , ROM 40 and RAM 41 are also components of the optical information reader 1 .

A lighting unit 4 is provided on the front side of the housing 2 . The illumination unit 4 is a part for illuminating at least the code of the work W by irradiating light toward the front of the optical information reading device 1 . As also shown in FIG. 3, the lighting unit 4 includes a first lighting unit 4a made up of a plurality of light emitting diodes (LEDs: Light Emission Diodes), a second lighting unit 4b made up of a plurality of light emitting diodes, and a first lighting unit 4b made up of a plurality of light emitting diodes. and an illumination drive unit 4c including an LED driver or the like for driving the unit 4a and the second illumination unit 4b.The first illumination unit 4a and the second illumination unit 4b are individually driven by the illumination drive unit 4c, The lighting drive unit 4c is connected to the processor 20, and is controlled by the processor 20. The first lighting unit 4c can be turned on and off. One of 4a and the second lighting unit 4b may be omitted.

As shown in FIG. 2, a camera 5 is provided in the central portion of the front side of the housing 2 . The direction of the optical axis of the camera 5 substantially coincides with the direction of light emitted by the illumination section 4 . The camera 5 is a part that photographs the code and generates a read image. As shown in FIG. 3, the camera 5 includes an imaging device 5a for receiving reflected light from a code attached to the work W and illuminated by the illumination unit 4, an optical system 5b having a lens and the like, an AF and a module (autofocus module) 5c. Light reflected from the coded portion of the workpiece W is incident on the optical system 5b. image.

The imaging device 5a is an image sensor comprising a light-receiving device such as a CCD (charge-coupled device) or CMOS (complementary metal oxide semiconductor) that converts a code image obtained through the optical system 5b into an electric signal. The image sensor 5a is connected to a processor 20, and the electric signal converted by the image sensor 5a is input to the processor 20 as read image data. The AF module 5c is a mechanism that performs focusing by changing the position and refractive index of a focusing lens among the lenses that make up the optical system 5b. The AF module 5 c is also connected to and controlled by the processor 20 .

As shown in FIG. 2, a display portion 6 is provided on the side surface of the housing 2. As shown in FIG. The display unit 6 is composed of, for example, an organic EL display, a liquid crystal display, or the like. The display unit 6 is connected to the processor 20, and can display, for example, the code imaged by the imaging unit 5, the character string that is the result of decoding the code, the reading success rate, the matching level, and the like. The reading success rate is the average reading success rate when the reading process is executed multiple times. The matching level is a reading margin indicating how easily a successfully decoded code can be read. This can be obtained from the number of error corrections that occur during decoding, and can be represented by a numerical value, for example. The fewer error corrections, the higher the matching level (reading margin), while the more error corrections, the lower the matching level (reading margin).

A power cable (not shown) for supplying power to the optical information reader 1 from the outside is connected to the power connector 7 . Signal lines 100 a and 101 a for communicating with the computer 100 and PLC 101 are connected to the signal line connector 8 . The signal line connector 8 can be composed of, for example, an Ethernet connector, a serial communication connector such as RS232C, a USB connector, or the like.

An indicator 9 is provided on the housing 2 . The indicator 9 is connected to the processor 20 and can consist of a light emitter such as a light emitting diode. The operating state of the optical information reader 1 can be notified to the outside by the lighting state of the indicator 9 .

A pair of aimer light irradiation units 10 are provided on the front side of the housing 2 so as to sandwich the camera 5 therebetween. As shown in FIG. 3, the aimer light irradiation unit 10 includes an aimer 10a composed of a light emitting diode or the like, and an aimer driving unit 10b for driving the aimer 10a. The aimer 10a irradiates light (aimer light) toward the front of the optical information reading device 1 to indicate the imaging range and the center of the visual field of the camera 5, the guideline of the optical axis of the illumination unit 4, and the like. . Specifically, the aimer 10a irradiates visible light of a color (for example, red or green) different from ambient light toward the range of the imaging field of view of the camera 5, and the surface irradiated with the visible light is visible to the naked eye. Form a visible landmark. The mark may be various figures, symbols, characters, or the like. The user can also set the optical information reader 1 by referring to the light emitted from the aimer 10a.

As shown in FIG. 2, the side surface of the housing 2 is provided with operation buttons 11 and 12 that are used when setting the optical information reader 1 and the like. The operation buttons 11 and 12 include, for example, select buttons and enter buttons. In addition to the operation buttons 11 and 12, for example, touch panel type operation means may be provided. The operation buttons 11 and 12 are connected to a processor 20 so that the processor 20 can detect the operation states of the operation buttons 11 and 12. FIG. By operating the operation buttons 11 and 12, it is possible to select one of a plurality of options displayed on the display unit 6 and to confirm the selected result.

(hand-held optical information reader)
In the above example, the optical information reader 1 is of a stationary type, but the present invention is applicable to devices other than the stationary optical information reader 1 . FIG. 4 shows a handheld optical information reader 1A, and the present invention can also be applied to a handheld optical information reader 1A as shown in this figure.

A housing 2A of the hand-held optical information reader 1A has a vertically elongated shape. The orientation of the optical information reader 1A when used is not limited to the illustrated orientation, and can be used in various orientations. is doing.

A display portion 6A is provided on the upper portion of the housing 2A. The display section 6A is configured in the same manner as the display section 6 of the stationary optical information reader 1. FIG. A lower portion of the housing 2A is a grip portion 2B for a user to grip during operation. The grip part 2B is a part that a general adult can hold with one hand, and its shape and size can be freely set. By holding the grip portion 2B, the optical information reader 1A can be carried and moved. In other words, this optical information reader 1A is a portable terminal device, and can be called a handy terminal, for example.

Similar to the stationary optical information reader 1, the hand-held housing 2A also accommodates an illumination unit, a camera, an aimer light irradiation unit, a processor, a storage unit, a ROM, a RAM, etc. (not shown). . The optical axis of the illumination unit, the optical axis of the camera, and the optical axis of the aimer light irradiation unit are directed obliquely upward from the vicinity of the upper end of the housing 2A. A buzzer (not shown) is also provided in the handheld housing 2A.

A plurality of operation buttons 11A and trigger keys 11B are provided on the grip portion 2B and its vicinity. The operation button 11A is the same as the operation button 11 of the stationary optical information reader 1 and the like. When the user presses the trigger key 11B with the tip (upper end) of the optical reader 1A facing the workpiece W, the tip of the optical reader 1A emits an aimer beam. When the user adjusts the direction of the optical reader 1A while visually observing the aimer light reflected on the surface of the work W, and aligns the aimer light with the code to be read, the code is automatically read and decoded. . When the reading is completed, the buzzer emits a completion notification sound.

As an example of use of the handheld optical information reader 1A, there is an example of using the handheld optical information reader 1A for picking work in a distribution warehouse. For example, when shipping ordered products from a distribution warehouse, the necessary products are picked from the product shelves in the product warehouse. This picking operation is performed by a user holding an order slip with a code written thereon, after moving to the product shelf, while comparing the code of the order slip with the code attached to the product or the product shelf. In this case, the hand-held optical information reader 1A alternately reads the code of the order slip and the code attached to the product or the product shelf.

(Processor configuration)
The following description is common to the stationary optical information reader 1 and the handheld optical information reader 1A, and can be applied to either apparatus 1 or 1A unless otherwise specified. As shown in FIG. 3, the processor 20 includes a CPU core 21, a DSP core 22, and an AI chip 23. The CPU core 21, DSP core 22 and AI chip 23 are a so-called system-on-a-chip (SoC, SOC) and are mounted on one substrate. The DSP core 22 and the AI chip 23 do not have to be SoC, in which case they are not mounted on the same substrate, but that case is also included in the scope of the present invention.

A high-speed RAM 41 is connected to the CPU core 21 , DSP core 22 and AI chip 23 , and all of the CPU core 21 , DSP core 22 and AI chip 23 can access the RAM 41 . A ROM 40 is also connected to the processor 20 , and all of the CPU core 21 , DSP core 22 and AI chip 23 can access the ROM 40 .

The CPU core 21 is a general-purpose processor, and is a part that executes, for example, AF control, illumination control, camera control, decoding processing for read images, and the like. The DSP core 22 is a part that executes various filtering processes, for example, on the read image. The AI chip 23 is an integrated circuit dedicated to trial restoration of the read image by the neural network, and is specialized for executing the sum-of-products operation necessary for processing by the neural network at very high speed.

As shown in FIG. 5, the processor 20 operates an AF control unit 20a, an imaging control unit 20b, a filter processing unit 20c, a tuning execution unit 20d, a decoding processing unit 20f, an extraction unit 20g, a reduction unit 20h, an expansion unit 20i, and an image restoration unit. A part 20j is configured. The AF control unit 20a, the imaging control unit 20b, the filter processing unit 20c, the tuning execution unit 20d, the decoding processing unit 20f, the extraction unit 20g, the reduction unit 20h, and the expansion unit 20i are configured by arithmetic processing of the CPU core 21 or the DSP core 22. This is the part where On the other hand, the image restoration unit 20 j is a part configured by the AI chip 23 .

(Configuration of AF control unit)
The AF control unit 20a is a unit that controls the AF module 5c shown in FIG. 3, and is configured to be able to focus the optical system 5b by conventionally known contrast AF or phase difference AF.

(Configuration of imaging control unit)
The imaging control section 20b is a unit that adjusts the gain of the camera 5, controls the amount of light of the illumination section 4, and controls the exposure time (shutter speed) of the imaging element 5a. Here, the gain of the camera 5 is an amplification factor (also called magnification) when the brightness of the image output from the image sensor 5a is amplified by digital image processing. The amount of light of the illumination section 4 can be changed by controlling the first illumination section 4a and the second illumination section 4b separately. The gain, the amount of light of the illumination unit 4 and the exposure time are imaging conditions of the camera 5 .

(Configuration of filter processing unit)
The filtering unit 20c is a part that performs an image processing filter on the read image. The filter processing unit 20c executes a noise removal filter that removes noise included in the image generated by the camera 5, a contrast correction filter that corrects contrast, an averaging filter, and the like. The image processing filters executed by the filter processing unit 20c are not limited to noise removal filters, contrast correction filters, and averaging filters, and may include other image processing filters.

The filter processing unit 20c is configured to perform an image processing filter on a read image before image restoration, which will be described later. Further, the filter processing unit 20c is configured to perform an image processing filter on a read image before enlargement or reduction, which will be described later. Note that the filter processing unit 20c may be configured to execute the image processing filter on the read image after image restoration, or may be configured to execute the image processing filter on the read image after enlargement or reduction. may be configured to

(Configuration of Tuning Execution Unit)
The tuning execution unit 20d shown in FIG. 5 changes the imaging conditions of the camera 5 and the decoding conditions of the decoding process, repeats the imaging and decoding processes, and makes it easier to read the code calculated under each imaging condition and decoding condition. Based on the matching level indicating the size, the optimum imaging condition and decoding condition are determined, and a tuning process for setting the size of the code to be read is executed. Specifically, when setting the optical information reader 1 or 1A, the tuning execution unit 20d sets the imaging conditions such as the gain of the camera 5, the light amount and exposure time of the illumination unit 4, and the image processing conditions in the filter processing unit 20c. is changed to set various conditions (tuning parameters) so that the conditions are suitable for decoding. The image processing conditions in the filter processing unit 20c include image processing filter coefficients (filter strength), switching of image processing filters when there are a plurality of image processing filters, combination of different types of image processing filters, and the like. Appropriate imaging conditions and image processing conditions differ depending on the influence of external light on the workpiece W during transportation, the color and material of the surface to which the code is attached, and the like. Therefore, the tuning execution unit 20d searches for more appropriate imaging conditions and image processing conditions, and sets processing by the AF control unit 20a, the imaging control unit 20b, and the filter processing unit 20c.

The tuning execution unit 20d is configured to be able to acquire the size of the code included in the read image generated by the camera 5. FIG. When acquiring the size of the chord, first, the tuning execution unit 20d searches for the chord based on the feature quantity indicating the likeness of the chord. Next, the tuning execution unit 20d acquires the PPC, code type, code size, etc. of the searched code. The PPC, code type, code size, etc. are included in the code parameters or code conditions, so the tuning execution unit 20d is configured to be able to acquire the code parameters or code conditions of the searched code.

The two-dimensional code is constructed by randomly arranging a plurality of white modules and black modules. For example, QR code model 2 has modules from 21×21 to 177×177. In the case of the stationary optical information reader 1, by reading the code to be read in advance, the number of modules and PPC are limited, and the code size (pixel size) is limited to the number of modules x PPC. .

PPC is a code parameter that indicates how many pixels (picture elements) constitute one module among the modules that constitute the code. After identifying one module, the tuning execution unit 20d can obtain the PPC by counting the number of pixels that constitute one module.

The code type is the type of code such as QR code, data matrix code, Vericode, for example. Since each chord has its own characteristics, the tuning execution unit 20d can determine the chord type by, for example, the presence or absence of a finder pattern.

Also, the code size can be calculated from the number of modules arranged in the vertical or horizontal direction of the code and the PPC. The tuning execution unit 20d can obtain the code size by calculating the number of modules arranged vertically or horizontally in the code, calculating the PPC, and multiplying the number of modules arranged vertically or horizontally by the PPC.

(Structure of decoding processing unit)
The decoding processing unit 20f is a part that decodes black and white binarized data. Decoding can use a table showing the correspondence of the encoded data. Furthermore, the decoding processing unit 20f checks whether the decoded result is correct according to a predetermined check method. If an error is found in the data, correct data is calculated using the error correction function. The error correction function differs depending on the type of code.

In this embodiment, the decoding processing unit 20f decodes the code included in the read image after image restoration, which will be described later, but can also decode the code included in the read image before image restoration. The decoding processing unit 20f is configured to write the decoding result obtained by decoding the code into the decoding result storage unit 30b of the storage unit 30 shown in FIG.

(Structure of extraction part)
The extraction unit 20g is a part that extracts a code candidate area that is highly likely to contain a code from the read image generated by the camera 5. FIG. A chord candidate region can be extracted based on a feature amount indicating chord-likeness. In this case, the feature amount indicating chord-likeness serves as information for specifying a chord. For example, the extraction unit 20g can acquire a read image and search for a code in the acquired read image based on a feature amount indicating code-likeness. Specifically, it is searched for whether or not there is a portion having a predetermined or more feature amount indicating code-likeness in the acquired read image, and as a result, a portion having a feature amount indicating code-likeness is searched. If possible, an area including that part is extracted as a code candidate area. The code candidate region may include regions other than code, but includes at least a portion that is more likely to be code than a predetermined number. Note that the code candidate region is a region in which there is a high possibility that a code exists, and therefore, there may be a case where the code is not included in the region.

The extraction unit 20g can also use the positional information of the aimer beam emitted from the aimer beam irradiation unit 10 as information for identifying the code, and extract the code candidate area. In this case, the extraction unit 20g identifies a portion corresponding to the mark formed by the aimer light irradiation unit 10 from the read image, and extracts the identified portion as a code candidate area. That is, as described above, since the aimer light is used to indicate the imaging range and the center of the field of view of the camera 5, the mark formed by the aimer light should be aligned with the center of the field of view of the camera 5 in advance. It is possible. In particular, in the case of the hand-held optical information reader 1A, the user aligns the aimer light with the code to be read, so there is a high probability that the mark formed by the aimer light overlaps the code. That is, by aligning the portion corresponding to the mark formed by the aimer light irradiation unit 10 with the center of the field of view of the camera 5, when the extraction unit 20g extracts an area including the center of the field of view of the camera 5, the area is a code. is likely to exist. In this case, the portion corresponding to the mark formed by the aimer light irradiation unit 10 does not have to be completely aligned with the center of the field of view of the camera 5, and may be slightly shifted in the vertical or horizontal direction of the field of view. Also, since the code has a predetermined size, the region extracted by the extraction unit 20g is not only the center of the field of view of the camera 5, but also a region of a predetermined size including the center of the field of view.

The extraction unit 20g can also specify the central portion of the read image and extract the central portion as the code candidate area. For example, by obtaining in advance the center of the field of view of the camera 5 as information for specifying the code, the portion that will be the center of the field of view of the camera 5 can be specified on the read image. In particular, in the case of the hand-held optical information reader 1A, since the central portion specified on the read image corresponds to the mark formed by the aimer light irradiation section 10, the code may exist in the central portion. This is an area with high potential.

The extraction unit 20g can be configured to accept a user's designation of a predetermined portion from the read image, and can extract the designated portion as a code candidate region. That is, when the user designates an area of arbitrary size at an arbitrary position (coordinates) on the read image, the extraction unit 20g extracts a predetermined area from the read image based on the information on the coordinates and size of the position. Accepts part specification. The extraction unit 20g extracts the received portion as a code candidate region. For example, in the case of a stationary optical information reader 1, as shown in FIG. The work W does not always exist in the center of the width direction of the belt conveyor B, but the work W may exist at the end. In this case, the conveying belt conveyor B By specifying the part corresponding to the end of , it is possible to accurately extract the area where the code is likely to exist. In addition, there are cases where the code is displayed near the edge of a large workpiece W, away from the center. By doing so, it is possible to accurately extract the region where the code is likely to exist.

(Structure of storage unit)
The storage unit 30 shown in FIG. 3 can be composed of a readable and writable storage device such as an SSD (solid state drive), but each of the storage units 30a, 30b, 30c, and 30d shown below is the above storage. It may be provided in the ROM 40 instead of the device. In other words, this embodiment also includes a configuration in which the ROM 40 includes the image data storage unit 30a, the decoding result storage unit 30b, the parameter set storage unit 30c, and the neural network storage unit 30d. The image data storage section 30a is a section that stores the read image generated by the camera 5. FIG. The decoding result storage unit 30b is a part that stores the decoding result of the code executed by the decoding processing unit 20f. The parameter set storage section 30c is a section that stores various conditions set as a result of tuning executed by the tuning execution section 20d and various conditions set by the user. The neural network storage unit 30d is a part that stores the structure and parameters of a trained neural network, which will be described later.

(Image restoration using neural networks)
The optical information reader 1, 1A has an image restoration function of restoring the read image acquired by the camera 5 using a neural network. In this embodiment, instead of performing machine learning in the optical information reader 1, 1A, the optical information reader 1, 1A holds in advance the structure and parameters of the neural network that has already completed learning, and It is different from conventional optical information reading devices in that it is configured to perform image restoration using a neural network configured with stored structures and parameters.

As shown in FIG. 6, the neural network has an input layer to which input data (image data in this example) is input, an output layer to output output data, and an intermediate layer provided between the input layer and the output layer. have. For example, a plurality of intermediate layers can be provided, thereby forming a multi-layered neural network.

(Neural network learning)
First, the basic procedure for neural network learning will be described based on the flow chart shown in FIG. Learning of the neural network can be performed using a computer prepared for learning other than the optical information reading devices 1 and 1A, but may be performed using a general-purpose computer other than for learning. Also, a conventionally well-known method may be used for the learning method of the neural network.

In step SA1 after the start, data of defective images prepared in advance are read. A defective image is an image having a portion inappropriate for code reading, and can be exemplified by the image shown on the left side of FIG. The inappropriate part is, for example, a dirty part, a colored part, or the like. After that, the process advances to step SA2 to read the ideal image data prepared in advance. The ideal image is an image suitable for reading the code, and can be exemplified by the image shown on the right side of FIG. A defective image and an ideal image are paired, and a plurality of such pairs are prepared.

At step SA2, a loss function is calculated to find the difference between the defective image and the ideal image. At step SA3, the parameters of the neural network are updated by reflecting the difference obtained at step SA2.

At step SA4, it is determined whether or not the machine learning completion condition is satisfied. The machine learning completion condition can be set based on the difference obtained in step SA2. For example, if the difference is a predetermined value or less, it can be determined that the machine learning completion condition is satisfied. If it is determined YES in step SA4 and the machine learning completion condition is satisfied, the machine learning is terminated. On the other hand, if it is determined NO in step SA4 and the machine learning completion condition is not satisfied, the process returns to step SA1, another defective image is read, and then the process proceeds to step SA2 to pair with the other defective image. Load another ideal image that is

In this manner, a learned neural network can be generated in advance by performing machine learning on a plurality of defective images and a plurality of ideal images corresponding to the plurality of defective images. By holding the structure and parameters of the trained neural network in the optical information reader 1, 1A, the trained neural network can be constructed in the optical information reader 1, 1A. That is, when machine learning is performed, a huge number of defective images and ideal images are input to the neural network, so that an extremely high computing power is required. It is difficult to secure such a high computing power in the optical information reading device 1, 1A because it may not be possible to satisfy the demand for miniaturization and weight reduction of the device. As in this example, machine learning is performed outside the optical information reader 1, 1A, and the structure and parameters of the neural network obtained as a result are held in the optical information reader 1, 1A. Image restoration by a neural network becomes possible while reducing the size and weight of the information reading device 1, 1A.

FIG. 9 is a conceptual diagram of a convolutional neural network (CNN) configured as described above. "Convolution" extracts the features of the input image. This layer consists of convolution operations such as image processing filters. The weight of the filter is called a kernel, and the feature quantity is extracted according to the kernel. A convolutional layer generally has a plurality of different kernels, and the number of maps (D) increases according to the number of kernels. The kernel size can take, for example, 3×3, 5×5, and so on.

In "pooling", the reduction process is performed to combine the reactions of each kernel. "Deconvolution" reconstructs an image from feature values using a deconvolution filter. "Unpooling" sharpens the reaction value by performing expansion processing.

The structure and parameters of the trained neural network are stored in the neural network storage section 30d of the storage section 30 shown in FIG. The neural network structure includes the number of intermediate layers (D) provided between the input layer and the output layer, the number of image processing filters, and the like. The parameters of the neural network are the parameters set in step SA3 of the flow chart shown in FIG.

Further, the neural network storage unit 30d can store the structures and parameters of a plurality of neural networks having different numbers of layers or different numbers of image processing filters. In this case, the structures and parameters of a plurality of neural networks and code conditions can be stored in the neural network storage unit 30d in association with each other. For example, the structure and parameters of the neural network for constructing the first neural network are associated with a first code condition corresponding to the first neural network. When making the association, the number of layers or filters of the first neural network can be prespecified optimally by the first code condition, especially the first PPC, so this relationship be associated so that Similarly, the structure and parameters of a second neural network, different from the first neural network, are associated with a second code condition, different from the first code condition.

When the structures and parameters of a plurality of neural networks are stored in the neural network storage unit 30d, the tuning execution unit 20d operates as follows when setting the optical information reader 1, 1A. That is, when setting the code conditions included in the read image generated by the camera 5, the tuning execution unit 20d reads the structure and parameters of the neural network associated with the set code conditions from the neural network storage unit 30d and operates them. Identify as a neural network to use from time to time. For example, when the tuning execution unit 20d sets the first code condition as the code condition included in the read image, the structure and parameters of the first neural network associated with the first code condition are set to the neural network. Read from the storage unit 30d. Then, the structure and parameters of the first neural network are specified as the neural network to be used during operation of the optical information reader 1, 1A. As a result, it is possible to try to restore the read image with the optimal neural network structure and parameters for the code conditions, so that reading accuracy can be improved.

In learning the neural network, it is also possible to machine-learn a defective image and an ideal image in which the pixel resolution of the modules constituting the code is within a specific range. The pixel resolution of a module can be represented, for example, by the number of pixels (PPC) of one module that constitutes the code, and machine learning of the neural network may be performed only with defective images and ideal images in which the PPC is within a specific range. .

The specific range includes pixel resolutions at which the reading image has a high restoration effect and excludes pixel resolutions at which the reading image restoration effect hardly improves. can be set as a premise. As a result, the processing speed can be improved as a neural network structure specialized for restoration of images containing codes.

The specific range of the number of pixels can be, for example, 4 PPC or more and 6 PPC or less, but it may be less than 4 PPC or more than 6 PPC. In addition, for example, by limiting the specific range to 4 PPC or more and 6 PPC or less for machine learning, it is possible to eliminate unnecessary scale fluctuations from the image used for learning, and it is also possible to optimize the trained neural network structure. .

(Repair attempt)
As shown in FIG. 5, the processor 20 of the optical information reader 1, 1A is provided with an image restoration section 20j configured by an AI chip 23. As shown in FIG. The image restoration unit 20j reads the structure and parameters of the neural network stored in the neural network storage unit 30d of the storage unit 30, and uses the read structure and parameters of the neural network to restore the neural network in the optical information reading devices 1 and 1A. Configure. The neural network configured in the optical information reader 1, 1A is the same as the trained neural network learned according to the procedure shown in the flow chart of FIG.

The image restoration unit 20j inputs the read image generated by the camera 5 to the neural network, and attempts restoration of the read image according to the structure and parameters of the neural network stored in the neural network storage unit 30d. Then, the decoding processing unit 20f executes decoding processing on the restored read image.

The filter processing unit 20c is configured to perform an image processing filter on the read image generated by the camera 5 before the image restoration unit 20j inputs the read image to the neural network. As a result, it is possible to perform appropriate image processing before attempting restoration by the neural network, and as a result, restoration of the read image becomes more accurate.

The image restoration unit 20j may attempt image restoration for all the read images to be decoded by the decode processing unit 20f, or the image restoration unit 20j may attempt image restoration only for some of the read images to be decoded by the decode processing unit 20f. 20j may attempt image restoration. For example, the decoding processing unit 20f executes the decoding process on the read image before the restoration is attempted, and determines whether or not the decoding is successful. If the decoding is successful, it means that the read image does not need restoration, and the result is output as it is. On the other hand, if the decoding fails, the image restoration unit 20j tries to restore the read image, and the decode processing unit 20f executes decoding processing on the restored read image.

(Input partial image)
The image that the image restoration unit 20j inputs to the neural network may be the entire read image generated by the camera 5. However, the larger the size of the input image, the greater the amount of computation by the neural network. 20, the computational load becomes heavy, and the processing speed may be lowered. Specifically, in a fully convolutional network (FCN) type neural network as shown in FIG. Since it is proportional to the square of the size of the code, the amount of computation is also proportional to the square of the size of the code.

On the other hand, it can be said that there are no cases where the code exists in the entire read image generated by the camera 5. During normal operation, the code exists only in a partial area of the read image, and only in that area. Reading accuracy can be improved if restoration by a neural network can be performed.

Therefore, the image input to the neural network by the image restoration unit 20j can be a part of the read image generated by the camera 5, that is, a partial image containing the code. Specifically, the image restoration unit 20 j cuts out a partial image corresponding to the code candidate area extracted by the extraction unit 20 g from the read image generated by the camera 5 . The image restoration unit 20j inputs the clipped partial image to the neural network to attempt restoration of the partial image according to the structure and parameters stored in the neural network storage unit 30d. Then, the decoding processing unit 20f executes decoding processing on the restored partial image.

Since there are almost no cases where the code exists in the entire read image as described above, the size of the partial image corresponding to the code candidate area is smaller than the size of the read image. As a result, the size of the image to be input to the neural network is reduced, so that the computational load on the processor 20 is reduced, and the processing speed is increased. In addition, since the partial image corresponds to an area where the code is highly likely to exist, it is possible to create an image containing the information required for reading, that is, the entire code. Since the image containing the entire code is input to the neural network, a decrease in reading accuracy is suppressed.

The image restoration unit 20j can determine the input size of the partial image to the neural network based on the code size acquired by the tuning execution unit 20d. As described above, the tuning execution unit 20d can acquire the code size. The image restoration unit 20j makes the input size of the partial image to be input to the neural network larger than the code size obtained by the tuning execution unit 20d by a predetermined amount.

That is, inputting the partial image to the neural network can reduce the computational load, but on the other hand, if, for example, the code is rotated or the position of the code cannot be detected precisely, the code in the partial image cannot be detected. It is also possible that part of the data is missing and the decoding process fails. In this embodiment, since the input size to the neural network is not the same as the code size obtained by the tuning execution unit 20d, but is larger than the code size by a predetermined amount, it is possible to prevent a part of the code from being missing in the partial image. . By making the code size larger than the code size by a predetermined amount, it is possible to give an upper limit to the input size to the neural network, thereby preventing the computational load from becoming heavy. The "predetermined amount" means, for example, that the horizontal (or vertical) length of the partial image input to the neural network is 1.5 times or more the horizontal (or vertical) length of the code acquired by the tuning execution unit 20d. , or 2.0 times or more, or 3.0 times or more.

When the image restoration unit 20j determines the input size of the partial image to be input to the neural network, the number of pixels in one module constituting the code acquired by the tuning execution unit 20d and the number of modules arranged in the vertical or horizontal direction of the code are used. can be determined based on

Also, the tuning execution unit 20d can set the code conditions as described above. The image restoration unit 20j can also set the size of the read image to be input to the neural network based on the code conditions set by the tuning execution unit 20d. For example, when the image restoration unit 20j acquires the code size among the code conditions set by the tuning execution unit 20d, a partial image having a size larger than the acquired code size by a predetermined amount is input to the neural network. If the code size is large, the size of the partial image to be input to the neural network will be large, while if the code size is small, the size of the partial image to be input to the neural network will be small. In other words, the size of the partial image input to the neural network can be changed according to the code conditions.

(reduction/enlargement function)
Here, in order to fully capture the features of the code in the convolutional neural network as shown in FIG. 9 and restore the image to be suitable for decoding, it is necessary to extract feature quantities within a range that covers a certain number of modules. be. For example, if one feature value obtained from the neural network is a value extracted from a range of 6×6 modules, and one module is an image captured with 20 pixels, it is aggregated from a range of 120×120 pixels. A neural network must be designed to calculate feature values that In other words, the wider the pixel range to be covered, the deeper the layer of the neural network must be. For example, to aggregate the feature values from the 120×120 pixel range described above, six convolution layers are required. Become.

However, since the code is composed of white modules and black modules arranged randomly, the relationship between pixel values over a wide range is weak. The effect is hardly improved. Such features are referred to herein as narrow range features.

In addition, the arrangement of fixed module patterns such as the finder pattern included in the code and the rectangular shape of the entire code can be wide-range features. It is also possible to recover satisfactorily using narrow-range features alone, without A wide range feature can be defined as a feature having a fixed shape over a wide range.

Therefore, when learning the neural network shown in FIG. 7, an image in which the PPC of the code is within a specific range can be used. As a result, it is possible to improve the processing speed as a neural network structure specializing in restoration of images including codes, but there is a concern that the restoration effect for read images in which the PPC is outside the specific range will be reduced. When learning the neural network, an image in which the PPC of the code is outside the specific range may be used.

The optical information reading apparatuses 1 and 1A of the present embodiment are provided with functions for reducing/enlarging the read image so that the PPC of the read image is within a specific range. As shown in FIG. 5, the processor 20 includes a reducing section 20h and an enlarging section 20i. The reduction unit 20h is a unit that generates a read image that is reduced so that the pixel resolution of the modules that constitute the code in the read image generated by the camera 5 is within the above-described specific range. is a portion for generating a read image that is enlarged so that the pixel resolution of the modules constituting the code in the read image generated by the camera 5 is within the above-described specific range.

The reducing unit 20h and the enlarging unit 20i, for example, determine whether the PPC of the code in the read image generated by the camera 5 is outside a specific range (4 PPC or more and 6 PPC or less). While shrinking or enlarging is not executed, if it is outside the specified range, shrinking or enlarging is executed. By executing reduction or enlargement, the PPC of the code in the partial image input to the neural network by the image restoration unit 20j falls within a specific range.

The image restoration unit 20j inputs the read image, which has been enlarged or reduced within a specific range, to a neural network configured with the structure and parameters stored in the neural network storage unit 30d. Attempt to repair the scanned image. Then, the decoding processing unit 20f executes decoding processing on the restored read image.

The extracting unit 20g searches for codes in the read image enlarged by the enlarging unit 20i or the read image reduced by the reducing unit 20h, and designates an area including the searched code as a code candidate area where there is a high possibility that the code exists. It may be configured to extract. In this case, the image restoration unit 20j inputs the partial image corresponding to the code candidate region extracted by the extraction unit 20g to the neural network configured with the structure and parameters stored in the neural network storage unit 30d. Attempt to inpaint the read image according to the structure and parameters.

When setting the optical information reading apparatus 1, 1A, the tuning execution unit 20d generates a plurality of read images (enlarged images) with different enlargement ratios and a plurality of read images (reduced images) with different reduction ratios. can be generated. The tuning execution unit 20d inputs a plurality of generated read images (enlarged images or reduced images) to the neural network, attempts to restore each read image, executes decoding processing on each restored read image, A reading margin indicating ease of reading a code is obtained. The tuning executing unit 20d specifies the enlargement ratio or reduction ratio of the read image with the obtained reading margin higher than a predetermined value as the enlargement ratio or reduction ratio to be used during operation. When the tuning execution unit 20d specifies the enlargement ratio or reduction ratio, the reduction unit 20h and the enlargement unit 20i read at the enlargement ratio or reduction ratio specified by the tuning execution unit 20d during operation of the optical information reader 1 or 1A. Enlarge or reduce the image.

That is, for example, if the specific range has a certain extent, it is conceivable that the reading margin will change even within the specific range if the enlargement ratio or the reduction ratio is changed. Since the enlargement ratio or reduction ratio of the read image during operation can be specified so that the read margin obtained by the tuning execution unit 20d is higher than a predetermined value, the processing speed and reading accuracy are improved.

After obtaining the reading slack for each read image, the tuning executing unit 20d selects the enlargement ratio or reduction ratio of the read image with the highest reading slack among the plurality of obtained reading slacks as the enlargement ratio to be used during operation. Or it can be specified as a reduction ratio. Thereby, the processing speed and reading accuracy can be further improved.

Further, by utilizing the fact that the read image can be reduced and enlarged by the reduction unit 20h and the enlargement unit 20i, the types of parameter sets can be increased. A plurality of parameter sets with different reduction ratios by the reduction unit 20h and a plurality of parameter sets with different enlargement ratios by the expansion unit 20i can be generated, and these parameter sets can be stored in the parameter set storage unit 30c of the storage unit 30. FIG. For example, when a plurality of reduction ratios such as 1/2, 1/4, and 1/8 can be set by the reduction unit 20h, a parameter set with a reduction ratio of 1/2, a parameter set with a reduction ratio of 1/4, and a reduction ratio A parameter set with a rate of 1/8 can be stored in the parameter set storage unit 30c. In addition, when the enlargement ratio by the enlargement unit 20i can be set multiple times, such as 2 times, 4 times, and 8 times, a parameter set for the enlargement ratio of 2 times, a parameter set for the enlargement ratio of 4 times, and a parameter set for the enlargement ratio of 8 times. A parameter set can be stored in the parameter set storage unit 30c. Then, any one parameter set can be applied from the plurality of parameter sets stored in the parameter set storage unit 30c.

(Image Restoration Filter)
The image restoration unit 20j may be a part that executes an image restoration filter that tries to restore the read image using a neural network. The image restoration filter restores the read image according to the structure and parameters stored in the neural network storage unit 30d by inputting the read image to a neural network configured with the structure and parameters stored in the neural network storage unit 30d. It is a filter that tries to

When the image restoration filter is made executable, a setting unit 20e (shown in FIG. 5) for setting the image restoration filter can be provided. The setting unit 20e is configured to be able to receive the setting of the image restoration filter by the user. The setting unit 20e generates and displays a user interface that allows the user to select one of "application of the image restoration filter" and "no application of the image restoration filter" when setting the optical information reader 1 or 1A, for example. It can be configured to be displayed on the unit 6 and accept selection by the user. Therefore, the image restoration unit 20j applies the image restoration filter set by the setting unit 20e to the read image to attempt restoration of the read image.

The setting unit 20e may be included in the tuning execution unit 20d, or may be configured separately from the tuning execution unit 20d. When the setting unit 20e is included in the tuning execution unit 20d, it becomes possible to set parameters related to the image restoration filter during tuning. The parameters relating to the image restoration filter can include "application of the image restoration filter" and "non-application of the image restoration filter". “Application of the image restoration filter” means setting the image restoration filter to be executable, and “non-application of the image restoration filter” means not setting the image restoration filter.

The parameters related to the image restoration filter are also information related to the image restoration filter, and in this case, a parameter set including information related to setting of the image restoration filter can be stored in the parameter set storage unit 30c. Since the parameters related to the image restoration filter include "application of the image restoration filter" and "non-application of the image restoration filter", the parameter set stored in the parameter set storage unit 30c is set so that the image restoration filter can be executed. and a second parameter set that does not set the image restoration filter. During operation of the optical information reader 1, 1A, one parameter set selected from the first parameter set and the second parameter set is applied. When the first parameter set is applied, the decode processing unit 20f performs decoding processing on the read image that has undergone image restoration by the image restoration filter, while the second parameter set is applied. In this case, the decode processing unit 20f performs the decoding processing on the read image for which the image restoration is not performed.

The parameter sets stored in the parameter set storage unit 30c also include items for setting code conditions included in the read image generated by the camera 5. FIG. In the items for setting the code conditions, the PPC, code type, code size, etc. acquired by the tuning execution unit 20d are set. For example, one parameter set includes PPC, code type, and code size as items for setting code conditions, gain of camera 5 as imaging conditions, light amount of illumination unit 4, exposure time, and filter processing unit 20c. An image processing filter type as an item of an image processing filter to be applied, a parameter of the image processing filter, and an application item of an image restoration filter can be included. For each of these items, the values set by the tuning execution unit 20d may be used as they are, or the user may arbitrarily change them.

(Example of tuning process procedure)
An example of the procedure of the tuning process performed by the tuning executing section 20d when setting the optical information reader 1, 1A will be specifically described with reference to the flowchart shown in FIG. At step SB1 after the start of the flow chart shown in FIG. 10, the tuning executing section 20d controls the illumination section 4 and the camera 5 to cause the camera 5 to generate a read image, and the tuning executing section 20d acquires the read image. At this time, the presence/absence and type of an image processing filter to be executed before the decoding process, and decoding process parameters for applying image restoration by a neural network are set to arbitrary parameters. Next, in step SB2, the tuning executing section 20d causes the decoding processing section 20f to execute decoding processing on the acquired read image.

After the decoding process, the process proceeds to step SB3, and the tuning execution section 20d determines whether or not the decoding process of step SB2 was successful. If NO is determined in step SB3 and the decoding process in step SB2 fails, that is, if the code cannot be read, the process advances to step SB4 to change the decoding process parameter to another parameter. Execute the decoding process again. If the decoding process fails with all the decoding process parameters, this flow is ended and the user is notified.

On the other hand, if YES is determined in step SB3 and the decoding process in step SB2 succeeds, the process proceeds to step SB5, where the tuning execution unit 20d determines code parameters (PPC, code type, code size, etc.). When the tuning execution unit 20d determines the code parameters, the neural network structure and parameters associated with the code parameters and stored in the neural network storage unit 30d are also determined (step SB6). At step SB6, a neural network is configured with the structure and parameters read from the neural network storage unit 30d.

At step SB7, the image restoration unit 20j attempts to restore the read image by inputting the read image to the neural network configured at step SB6, and the decode processing unit 20f decodes the restored read image. Execute the process. After that, the process proceeds to step SB8, and the tuning execution unit 20d evaluates the reading margin based on the decoding processing result of step SB7 and temporarily stores it.

At Step SB9, it is determined whether or not the decoding process has been completed with all the decoding process parameters. If NO is determined in step SB9 and execution of the decoding process has not been completed with all the decoding process parameters, the process proceeds to step SB10, changes the decoding process parameters to other parameters, and then proceeds to step SB7.

On the other hand, when it is judged as YES in step SB9 and execution of the decoding process is completed with all the decoding process parameters, the process proceeds to step SB11. At step SB11, the tuning execution unit 20d selects the decoding process parameter with the highest reading margin from all the decoding process parameters, and determines the selected decoding process parameter as the parameter to be applied during operation. Incidentally, in the tuning process, the imaging conditions and the like are also set to appropriate conditions.

(Decoding processing procedure before determination of reduction ratio and enlargement ratio)
Next, an example of the decoding processing procedure before determining the reduction rate and the enlargement rate will be specifically described with reference to the flowchart shown in FIG. The processing specified in the flowchart shown in FIG. 11 can be executed in step SB2 of the flowchart shown in FIG.

At step SC1 after the start of the flowchart shown in FIG. 11, the tuning execution unit 20d selects an arbitrary image processing filter from among a plurality of image processing filters. This image processing filter is a filter executed by the filter processing unit 20c. After that, in step SC2, the filter processing unit 20c executes the image processing filter selected in step SC1 on the read image.

Next, at step SC3, an image pyramid is created. The image pyramid consists of an original read image, an image reduced to 1/2 of the original read image, an image reduced to 1/4 of the original read image, an image reduced to 1/8 of the original read image, and so on. It is configured. Reduction of the read image is performed by the reduction unit 20h. The image pyramid consists of an original read image, an image obtained by enlarging the original read image by two times, an image obtained by enlarging the original read image by four times, an image obtained by enlarging the original read image by eight times, and so on. You can also Enlargement of the read image is performed by the enlarging section 20i. One of the reduced image and the enlarged image may be omitted.

After creating the image pyramid, the process advances to step SC4 to select an arbitrary read image from among the plurality of read images forming the image pyramid. This selected image may include the original scanned image that has not been scaled down or scaled up. At step SC5, the extraction unit 20g extracts a code candidate area in which a code is likely to exist from the read image selected at step SC4. Then, in step SC6, the image restoration unit 20j inputs the partial image corresponding to the code candidate area extracted in step SC5 to the neural network to attempt restoration of the read image. The size of the partial image input to the neural network is determined by the code conditions described above.

After the read image is restored, the process proceeds to step SC7 to execute positioning processing of the outline of the code and the modules forming the code. After the positioning process, the process proceeds to step SC8 to determine whether each module constituting the code is white or black. After determining whether the image is black or white, the process proceeds to step SC9, and the decoding processing unit 20f executes decoding processing on the restored read image. In decoding processing, for example, a method of restoring a character string from a 0-1 matrix of modules can be adopted.

Thereafter, the process proceeds to step SC10, and the tuning execution unit 20d determines whether or not the decoding process of step SC9 was successful. If NO is determined in step SC10 and the decoding process in step SC9 fails, the process proceeds to step SC11 to determine whether or not all the read images forming the image pyramid have been selected. If the determination in step SC11 is NO and all the read images forming the image pyramid have not been selected, the process proceeds to step SC4 to select another reduced read image or another enlarged image forming the image pyramid. The scanned image is selected, and the process proceeds to step SC5.

On the other hand, if YES is determined in step SC11 and all the read images forming the image pyramid have been selected, this flow is terminated and the conditions for successful decoding are stored.

(Decode processing procedure after determination of reduction ratio and enlargement ratio)
Next, an example of the decoding processing procedure after determining the reduction rate and the enlargement rate will be specifically described with reference to the flowchart shown in FIG. The processing specified in the flowchart shown in FIG. 12 can be executed during operation of step SB7 of the flowchart shown in FIG. 10 and the optical information reader 1, 1A.

At step SD1 after the start of the flow chart shown in FIG. 12, the filter processing section 20c executes the image processing filter selected at step SC1 of the flow chart shown in FIG. 11 on the read image. At this time, when the filter processing unit 20c executes the averaging filter on the read image as shown in the upper part of FIG. 13, the image after execution of the image processing filter as shown in the lower part of FIG. 13 is obtained. .

After that, the process advances to step SD2 to execute reduction/enlargement processing of the read image as necessary. Specifically, when the PPC of the code in the read image after execution of the image processing filter is within a specific range, reduction or enlargement is not executed, and when it is outside the specific range, reduction or enlargement is not executed. Execute and make sure the PPC is within a certain range. The upper part of FIG. 14 shows an example of a read image after reduction processing.

Next, in step SD3, the extracting unit 20g extracts code candidate areas in which codes are likely to exist from the read image reduced or enlarged in step SD2. The lower part of FIG. 14 shows an example of the chord candidate area extraction image. If the image is not reduced or enlarged in step SD2, the original read image is extracted in step SD3.

After that, in step SD4, the image restoration unit 20j inputs the partial image corresponding to the code candidate area extracted in step SD3 to the neural network and tries to restore the read image. FIG. 15 shows an example of a read image before restoration and an example of a read image after restoration. After the read image is restored, this flow ends through steps SD5 to SD7. Steps SD5-SD7 are the same as steps SC7-SC9 in the flow chart shown in FIG.

(Action and effect of the embodiment)
As described above, according to this embodiment, when the read image generated by the camera 5 is input to the neural network during operation of the optical information reader 1 or 1A, restoration of the read image is attempted. A read image after the attempted repair is obtained that has been repaired to be an image suitable for reading the code. Since decoding processing is executed on the read image after this restoration attempt, reading accuracy is improved.

Since the neural network used for restoration is composed of a structure and parameters obtained by previously machine-learning a plurality of defective images and a plurality of ideal images, the optical information reader 1, 1A Then, only the inference processing by the neural network needs to be performed without performing the learning processing, thereby lightening the computational load of the processor 20 . As a result, the handheld optical information reader 1A can be made smaller and lighter, and the processing speed of the stationary optical information reader 1 can be increased.

When a code candidate area is extracted from the read image during operation of the optical information reader 1 or 1A, a partial image corresponding to the code candidate area is input to the neural network to attempt restoration of the partial image. be done. As a result, the size of the image to be input to the neural network is reduced, so that the computational load on the processor 20 is reduced, and the processing speed is increased. In addition, since the partial image corresponds to an area where the code is highly likely to exist, it is possible to create an image containing the information required for reading, that is, the entire code. Since the image including the entire code is input to the neural network, a decrease in reading accuracy is suppressed.

In addition, when generating the structure and parameters of the neural network, machine learning is performed on defective images and ideal images in which the pixel resolution of the modules that make up the code is within a specific range, so it is possible to restore images containing codes. Processing speed can be improved as a specialized neural network structure. If the pixel resolution of the modules that make up the code in the read image is outside the specified range, the read image is enlarged or reduced so that it falls within the specified range before being input to the neural network. It enables restoration of images containing resolution codes.

(Other embodiments)
FIG. 16 is a diagram for explaining an example of the processing procedure during operation of the optical information reader 1 according to another embodiment. In this embodiment, the processor 20 configures a preprocessing section 200, a postprocessing section 201, a first decoding processing section 202, and a second decoding processing section 203 in addition to the image restoration section 20j. . The preprocessing section 200 is a section that executes a noise removal filter, a contrast correction filter, an averaging filter, etc. on the read image generated by the camera 5, similarly to the filter processing section 20c.

The first decoding processing unit 202 is a part that performs decoding processing on the read image generated by the camera 5, and the decoding processing by the first decoding processing unit 202 is also called first decoding processing. When the processor 20 has a plurality of cores (N cores), the first decoding processing unit 202 is composed of the cores 0 to i. The first decoding processing unit 202 may be configured with one core.

The second decoding processing unit 203 is a part that performs decoding processing on the restored image generated by the image restoration unit 20j, and the decoding processing by the second decoding processing unit 203 is also called second decoding processing. When the processor 20 has N cores, the second decoding processing unit 203 is composed of the cores i+1 to N−1. The second decoding processing unit 203 may be composed of one core.

As shown in FIG. 17, when the first decoding processing unit 202 is composed of core 0 to core i, each of core 0 to core i has a code in the read image generated by the camera 5. After extracting the code candidate regions with high probability, the code outline and positioning of the modules that compose the code are performed, and then each module that composes the code is white or black. is determined and decode processing is executed. A processing result is output to the post-processing unit 201 .

On the other hand, as shown in FIG. 17, when the second decoding processing unit 203 is composed of core i+1 to core N-1, each of core i+1 to core N-1 may detect the existence of a code from the read image. A code candidate region with a high value is extracted as a restoration code region, and a trigger signal is sent to the image restoration unit 20j so as to execute inference processing. In FIG. 17, two cores are described to execute one task, but one task may be executed by any one or more cores.

Specifically, when the cores N-2 and N-1 constituting the second decoding processing unit 203 extract the restoration code candidate region, they output a trigger signal (trigger 1) to the image restoration unit 20j, The image restoration unit 20j is caused to execute inference processing. When the image restoration unit 20j receives the trigger signal (trigger 1) sent from the cores N-2 and N-1, the portion corresponding to the restoration code candidate region extracted by the cores N-2 and N-1 An image is input to a neural network, and an inference process is executed to generate a restored image obtained by restoring the partial image (image restoration 1). The restored image generated in image restoration 1 is sent to core N-2 and core N-1. The cores N-2 and N-1 determine grid positions indicating the cell positions of the code based on the restored image generated in the image restoration 1, and execute decoding processing of the restored image based on the determined grid positions. do.

On the other hand, when the core i+1 and core i+2 constituting the second decoding processing unit 203 extract the restoration code candidate area, they output a trigger signal (trigger 2) to the image restoration unit 20j, and the image restoration unit 20j Let the inference process run. When the image restoration unit 20j receives the trigger signal (trigger 2) sent from the core i+1 and the core i+2, it inputs a partial image corresponding to the restoration code candidate region extracted by the core i+1 and the core i+2 to the neural network, An inference process is executed to generate a restored image obtained by restoring the partial image (image restoration 2). The restored image generated in image restoration 2 is sent to core i+1 and core i+2. Core i+1 and core i+2 determine grid positions indicating the positions of each cell of the code based on the restored image generated in image restoration 2, and decode the restored image based on the determined grid positions.

Thereafter, in the same manner, image restoration 3 is executed by trigger signals (trigger 3) sent from core N-2 and core N-1, and image restoration 3 is performed by trigger signals (trigger 4) sent from core i+1 and core i+2. Restoration 4 is executed, image restoration 5 is executed by a trigger signal (trigger 5) sent from core N-2 and core N-1, and image restoration is performed by a trigger signal (trigger 6) sent from core i+1 and core i+2. Restoration 6 is executed, and image restoration 7 is executed by a trigger signal (trigger 7) sent from core N-2 and core N-1.

As shown in this time chart, the inference processing by the image restoration unit 20j and the first decoding processing by the first decoding processing unit 202 are executed in parallel. For example, while the first decoding processing unit 202 continuously performs code region extraction, grid positioning, and decoding processing, the inference processing by the image restoration unit 20j is performed once or multiple times. Further, while the first decoding processing unit 202 continues the code region extraction, grid positioning, and decoding processing, the second decoding processing unit 203 extracts the recovery code region, outputs the trigger signal, grid positioning, and performs the second decoding. Processing is performed. In this way, the image restoration unit 20j can specialize in restoration of the partial image corresponding to the restoration code candidate area, so that the inference processing speed can be increased.

At least part of the period during which the second decoding processing section is executing the second decoding processing has a pause period during which the inference processing by the image restoration section 20j is not executed. That is, as shown in FIG. 17, after the image restoration 1 of the image restoration unit 20j is completed, the second decoding processing unit 203 starts the second decoding processing. A predetermined period is provided before the start, and this predetermined period is a pause period during which the image restoration section 20j does not perform image restoration. This pause period is a period during which the image restoration unit 20j does not perform inference processing. Note that the grid positioning processing time may be long or short depending on the restoration image to be processed. Although FIG. 17 shows an example in which the processing of each core is alternately executed, the processing of each core is not necessarily alternately executed depending on the processing time of grid positioning. In other words, trigger generation and grid positioning may be executed in order from the cores that are idle for processing.

As described above, the setting unit 20e is configured so that the user can select one of "application of the image restoration filter" and "non-application of the image restoration filter". Applying the image restoration filter means performing image restoration, and not applying the image restoration filter means not performing image restoration. It also corresponds to a part for setting whether or not the processing unit 203 executes the second decoding process. Since this setting is performed by the user, the setting unit 20e is configured to be able to receive the operation by the user. The setting form is not limited to the application or non-application of the image restoration filter, and may be, for example, the application or non-application of inference processing (restoration processing) or the application or non-application of processing by AI. The operation by the user can be, for example, the operation of a button or the like displayed on the user interface screen.

The optical information reading device 1 is operated after performing a tuning process for predetermining imaging conditions such as exposure time and gain, and settings such as an image processing filter. For example, when the workpiece W is a label, the code is clearly printed, so if appropriate conditions are set by the tuning process, a high reading rate can be achieved. However, in the case of workpieces W such as metal and resin, although optimal conditions are set for the tuning process, it may become difficult to read due to changes in the read image due to weak ambient light, etc. Due to the design, there may be cases where reading is not possible due to ultra-low contrast.

By executing the image restoration algorithm executed by the image restoration unit 20j of the present example in parallel with the decoding process without image restoration, it is possible to read the work W such as metal or resin while maintaining the conventional reading performance. Reading performance can be improved and reading stability can be enhanced. In addition, the work W that cannot be read by conventional optical design and algorithms can be read by applying the image restoration algorithm.

To give a specific example, in the process of reading a code printed on a sticker attached to a box in a certain factory, if there are no restrictions on the installation conditions of the optical information reader 1 and optimal installation is possible, Since stable operation is possible even with conventional reading performance, priority can be given to takt time by not applying the image restoration filter.

Also, in a certain factory, there is a process to read codes clearly printed on metal, and although the optimum conditions are determined in the tuning process at the time of reading setting, when actually operating, scratches on the metal surface, In some cases, reading may become unstable due to variations in reading position/angle, changes in the brightness of the surrounding environment, etc. In such a case, by applying an image restoration filter to improve the image quality, it becomes resistant to the above-described variations in conditions, and the reading performance is improved.

Also, some workpieces cannot be read by the conventional algorithm, but in such cases, applying an image restoration filter may make it possible to read them. For example, even if the code is severely damaged or the contrast is not clear, by applying an image restoration filter, the contrast and flaws can be restored and the code can be read.

Since the second decoding process executed when the image restoration filter is applied executes the decoding process on the restored image, it takes longer time than the first decoding process which executes the decoding process of the normal read image. is often long. Therefore, the first decoding processing section 202 can execute the first decoding processing in a shorter time than the time required for the second decoding processing by the second decoding processing section 203 . In this example, on the premise of this, the setting unit 20e can change the decode timeout time, which is the time limit for the decoding process.

Specifically, when the setting unit 20e is set to execute the second decoding process, the setting unit 20e is configured to be able to set a decoding timeout time longer than the time required to execute the second decoding process. When it is set that the second decoding process is not to be executed, a decoding timeout period shorter than the time required to execute the second decoding process can be set. As a result, during operation, when executing the second decoding process, it is possible to set a decoding timeout time longer than the time normally required for executing the second decoding process. can run. On the other hand, if it is set not to execute the second decoding process, generally only the first decoding process that can be decoded in a shorter time than the second decoding process is executed. A corresponding short decode timeout period can be set. By changing the decode timeout time in this way, it is possible to set the decode timeout time corresponding to each of the first decoding process and the second decoding process.

17, the first decoding processing unit 202 and the second decoding processing unit 203 are separated, but as shown in FIG. 18, the first decoding processing unit 202 and the second decoding processing unit 203 are combined. may be the decode processing unit 210 . The decoding processing unit 210 performs a second decoding process on the restored image in parallel with the first decoding process on the read image generated by the camera 5 . The part that executes the decoding process in this way can be arbitrarily divided on the hardware, or can be integrated. Hereinafter, an example in which the part that executes the decoding process is integrated into one will be described, but the present invention is not limited to this, and can be similarly applied to a case where the part is divided into a plurality of parts.

When the setting unit 20e sets not to execute the second decoding process, the decoding processing unit 210 allocates processing resources used for the second decoding process to the first decoding process, and executes the second decoding process. Then, the first decoding process is configured to be faster than when it is set. The resources for processing are, for example, memory usage areas, cores constituting a multi-core CPU, and the like. Specifically, when it is set not to execute the second decoding process, the number of cores in charge of the first decoding process is increased compared to when it is set to execute the second decoding process. Further, when it is set not to execute the second decoding process, the memory area used in the first decoding process is expanded compared to when it is set to execute the second decoding process. Increasing the number of cores and expanding the memory area may be used in combination, or only one of them may be performed. As a result, the performance of the multi-core CPU can be maximized and the speed of the first decoding process can be increased.

The decoding processing unit 210 extracts a code candidate area in which a code is likely to exist from the read image prior to image restoration. That is, there is a problem that if the image to be input to the neural network is large during image restoration using the neural network, the computation time becomes long. Also, from the viewpoint of speeding up the processing, it is desirable to reduce the number of decoding attempts as much as possible. On the other hand, in this example, a heat map-based search method is adopted instead of a line search-based method in order to ensure that the code candidate region can be extracted even if it takes some time. For example, the decoding processing unit 210 quantifies the feature amount of the code, generates a heat map in which the magnitude of the feature amount is assigned to each pixel value, and displays code candidates that are highly likely to exist on the heat map. Extract regions. As a specific example, there is a method of acquiring a characteristic portion (eg, finder pattern, etc.) of a two-dimensional code in an area that is relatively hot (having a large characteristic amount) in the heat map. When a plurality of characteristic portions are acquired, they can be extracted with priority and stored in the RAM 41, the storage unit 30, or the like.

After the decoding processing section 210 extracts the code candidate area, it outputs a partial image corresponding to the extracted code candidate area to the image restoration section 20j. The image restoration unit 20j inputs the partial image corresponding to the code candidate region extracted by the decoding processing unit 210 to the neural network, and performs inference processing to generate a restored image by restoring the partial image. Thereby, the calculation time can be shortened.

Further, the decoding processing unit 210 extracts a code candidate area having a size that can contain the code to be read set by the tuning process described above and having a high possibility that the code exists in the read image. The image may be reduced or enlarged to a predetermined size and then input to a neural network, and the restored image restored by the neural network may be decoded. By enlarging/reducing the extracted image to an image of a predetermined size and inputting it to the neural network, the PPC of the code image input to the neural network can be kept within a predetermined range. The image restoration effect can be drawn out stably. In addition, although the size of the code to be read may vary, since the size of the code that is finally input to the neural network can be fixed, the balance between high speed processing and ease of decoding can be achieved. can keep.

Flowcharts of the first decoding process and the second decoding process will be described in detail below with reference to FIG. At step SE1 after the start, the processor 20 causes the camera 5 to generate a read image and acquires the read image. The read image acquired in step SE1 is input to the decoding processing section 210. FIG. At step SE2, a first decoding process is executed without performing image restoration on the read image acquired at step SE1. If the first decoding process succeeds, the process advances to step SE4 to execute end processing, that is, decoding result output processing. If the first decoding process fails in step SE2, the process returns to step SE2 and the first decoding process is executed again. When a predetermined decode timeout time elapses, the first decoding process for the read image is stopped.

On the other hand, in the route from step SE1 to step SE5, the decode processing section 210 extracts the restoration code area from the read image. Specifically, a code candidate area having a size that can contain the code to be read set by the tuning process and having a high possibility that the code exists in the read image is extracted. At this time, if a plurality of restoration code regions are extracted, they are temporarily stored as candidate regions R1, R2, . An example of the extracted repair code region is shown in FIG. After that, the process proceeds to step SE6 and sets the value of k to 1.

At step SE7, the partial image corresponding to the candidate area Rk is resized. The resizing process is a process of reducing or enlarging the partial image corresponding to the restoration code area extracted in step SE5 to a predetermined size. With this resizing process, the size of an image to be input to a neural network, which will be described later, can be set to a predetermined size in advance. In the example shown in FIG. 20, the size of the partial image to be input to the neural network is 256 pixels×256 pixels. You can resize it to any size you want.

At step SE8, it is determined whether or not the black/white reversal setting is "both". In other words, image inpainting is only a mapping mathematically, and it is difficult to realize white-to-black, black-to-white, black-to-black, and white-to-white mappings with a single neural network. can be expensive. Therefore, black-and-white reversal processing is executed based on the property so that the image is printed in black on a white background. Depending on how the neural network is trained, the image may be printed in white on a black background. FIG. 20 shows an example of black-and-white reversal processing. Details of the black-and-white reversal processing will be described later.

If the determination in step SE8 is YES and the black/white inversion setting is set to "both", the process proceeds to step SE9, and the image after resizing is input to the image restoration section 20j to execute image restoration. After that, the process proceeds to step SE10, and the decoding processing unit 210 executes the second decoding process on the restored image. If the second decoding process succeeds in step SE10, the process proceeds to step SE4 to execute the end process. If the second decoding process fails in step SE10, the process proceeds to step SE11, and black and white reversal processing is performed on the resized image. After that, in step SE11, the image after black-and-white inversion processing is input to the image restoration section 20j to execute image restoration (see FIG. 20). Next, proceeding to step SE13, the decoding processing unit 210 performs the second decoding processing on the restored image. If the second decoding process succeeds in step SE13, the process proceeds to step SE4 to execute the end process. If the second decoding process fails in step SE13, the process proceeds to step SE14. At step SE14, it is determined whether or not the candidate area extracted at step SE5 is still present.

On the other hand, if NO is determined in step SE8 and the black/white inversion setting is not "both", the process proceeds to step SE15 to determine whether or not the black/white inversion setting is "OFF". If NO is determined in step SE15 and the black/white inversion setting is not "OFF", the process proceeds to step SE16 to invert the black/white of the resized image. After that, in step SE17, the image after black-and-white inversion processing is input to the image restoration section 20j to execute image restoration. Next, proceeding to step SE18, the decoding processing unit 210 performs the second decoding processing on the restored image. If the second decoding process succeeds in step SE18, the process proceeds to step SE4 to execute the end process. If the second decoding process fails in step SE18, the process proceeds to step SE14. At step SE14, it is determined whether or not there are still candidate areas, and if there are no candidate areas, the process proceeds to step SE4 to execute end processing.

If it is determined in step SE14 that there is still a candidate area, the process proceeds to step SE19, adds 1 to the value of k, and proceeds to step SE7. At step SE7, since the value of k is increased by one from the previous flow, the partial image corresponding to the next candidate region Rk is similarly resized. Step SE8 and subsequent steps are as described above. That is, if the decoding is successful in either the first decoding process or the second decoding process, the decoding result is output at the time of success, and the decoding process is not performed for the other candidate areas, and the read image of the next work W is picked up. and perform similar processing.

The post-processing unit 201 shown in FIG. 20 performs a process of synthesizing a partial image that has not been restored after resizing and a partial image that has been restored by the image restoring unit 20j after resizing. That is, in image restoration using a neural network, even though the code before restoration can be visually confirmed by humans, the image restoration may not go well or the cells may become rounded after the image restoration. For these reasons, if a partial image that has not been restored and a partial image that has been restored are superimposed at an appropriate ratio, subsequent decoding processing may be facilitated. The ratio of overlapping the partial image that has not been image-repaired and the partial image that has been image-repaired may be a predetermined fixed value or may be a value other than the fixed value. Based on the degree of stability, it may be determined dynamically at the time of reading or optimized by pre-evaluation (tuning). For example, in the preliminary evaluation, the user may be allowed to adjust the ratio using a scale bar or the like so that the adjustment can be made while presenting the synthesized image to the user.

(Bank switching function)
In the present embodiment, for example, the parameter set storage unit 30c can store a parameter set including parameters that configure the imaging conditions of the camera 5 and parameters that configure the processing conditions in the decoding processing unit 210. It is configured. A parameter set can be called a bank. A plurality of banks are provided, and different parameters are stored in each bank. For example, in the parameter set storage unit 30c, a plurality of imaging conditions and code conditions including the first imaging condition and code condition set by the tuning execution unit 20d and the second imaging condition and code condition are separately stored. is stored as a parameter set of

In this optical information reading device 1, from among the plurality of parameter sets stored in the parameter set storage unit 35c, one parameter set including the first imaging condition and the code condition is used to obtain the second imaging condition and the code condition. and vice versa. The parameter set switching can be performed by the processor 20, by the user, or can be configured to be performed by a switching signal from an external control device such as the PLC 101. FIG. When the user switches the parameter set, the user may operate a parameter set switching unit incorporated in the user interface, for example. By enabling the parameter set switching unit, the parameter set of the bank is used during operation of the optical information reader 1, and by disabling the parameter set switching unit, the parameter set of the bank is used. The set is not used when the optical information reader 1 is operated. That is, the parameter set switching unit is for switching from one parameter set to another parameter set.

When the decoding processing unit 210 of this example fails the second decoding processing for the restored image while operating using one parameter set, the decoding processing unit 210 switches to another parameter set, and then the image is captured under another condition. Image restoration and second decoding processing are performed on the read image. For example, when the decoding processing unit 210 detects that the second decoding process has failed, it switches to a parameter set other than the currently used parameter set, then executes image restoration and the second decoding process. When it detects that the second decoding process has failed, it switches to another parameter set that has not been used before, and then executes image restoration and second decoding process. When the parameter set is switched, not only the imaging condition of the camera 5 but also the code size to be read may be changed. In this case, the size when extracting the code candidate area where the code is likely to be present also changes. It will be. As a result, the reduction ratio or enlargement ratio of the image input to the neural network also changes, and the result of image restoration also changes.

(Supports black and white inversion)
Next, correspondence to white/black inversion of codes will be described with reference to FIGS. 21 and 22. FIG. FIG. 21 illustrates a case where two neural networks, namely, a neural network trained with a code printed in black on a white background and a neural network trained with a code printed in white on a black background, are used to handle black-and-white inversion of a code. It is a diagram. FIG. 21A inputs the read image (leftmost image) to a neural network trained with a black-on-white code and a neural network trained by a white-on-black code, and outputs decoded results from each neural network. This method is called a parallel execution method. Since the read image is an image of a black-printed code on a white background, decoding is successful in the restored image by a neural network trained with a black-printed code on a white background, but the decoded image is successfully decoded by a neural network trained with a white-printed code on a black background. Decoding fails on the restored image.

FIG. 21B first inputs the read image (the image at the left end) only to the neural network trained with the white-on-black code, and attempts to decode the restored image. Only when the decoding process fails, the read image is input to a neural network that has been trained with black-on-white code to generate a restored image. This method is a method of using another network after a failure.

FIG. 21C is a method of selecting a neural network for inputting a read image based on user settings. The case shown on the left side is a case where the read image is set to be an image of a black-printed code on a white background. In this case, the network selection module selects a neural network trained with black-on-white code. Then, the read image (image at the left end) is input to a neural network that has been trained with black-printed codes on a white background. On the other hand, the case shown on the right side is a case where the read image is set to be an image of a white-printed code on a black background. In this case, the network selection module selects a neural network trained with white-on-black code. Then, the read image (image at the left end) is input to a neural network that has been trained with white-printed code on a black background.

FIG. 22 is a diagram for explaining a case where only one neural network is used to handle code inversion. FIG. 22A shows a case where a black-and-white inverted image is generated by subjecting the read image (image at the left end) to black-and-white inversion processing. An image without white-black reversal processing and an image with black-white reversal processing are input to a neural network that has been trained with a code for black printing on a white background, respectively. Since the read image is an image of a white-printed code on a black background, the decoding process fails with the restored image by the neural network trained with the black-printed code on a white background. , the decoding process succeeds.

FIG. In the example shown in the upper part of 22B, the read image (leftmost image) is an image of a code printed in white on a black background. After that, a black-and-white reversed image is generated and input to a neural network that has learned black-on-white code, and the decoding process succeeds.

FIG. In the example shown below 22B, the read image (leftmost image) is an image of a code printed in black on a white background, so if it is input to a neural network trained with a code printed in white on a black background, the decoding process fails. After that, a black-and-white reversed image is generated and input to a neural network that has learned a white-on-black code, and the decoding process succeeds.

FIG. 22C is a method for determining whether or not to execute black-and-white reversal processing based on user settings. The case shown on the upper side is a case where the read image is set to be an image of a black-printed code on a white background. In this case, the black-and-white reversal processing module does not perform the black-and-white reversal processing, but directly inputs to the neural network that has learned the code for black printing on a white background. On the other hand, the case shown on the lower side is a case where the read image is set to be an image of a white-printed code on a black background. In this case, the black-and-white reversal processing module performs the black-and-white reversal processing. Then, the black-and-white reversed image is input to a neural network that has been trained with black-printed code on a white background.

(Relationship between contrast and matching level)
The matching level is used, for example, as a score during tuning, or for managing print quality/reading performance during operation. For example, the matching level is defined as a positive integer value from 0 to 100, and if it is 50 or higher, it can be designed so that the read rate is 100% when a read rate test is performed. Also, matching level 0 is the lowest value when reading is impossible, and matching level 1 is the lowest value when reading is possible.

FIG. 23 is a graph showing the relationship between the contrast and the matching level. The solid line shows the relationship between the contrast of the image (repaired image) restored by the image restoration unit 20j and the matching level. shows the relationship between the contrast of an image (read image) without image restoration and the matching level. As is clear from this graph, the matching level of the repaired image is generally higher than the matching level of the read image.

(Matching level calculation process)
The matching level is set for each algorithm, generally one algorithm per code type. However, in this embodiment, two algorithms (an image restoration algorithm and an image restoration algorithm not executed) are executed with one code type by installing the image restoration function. Of these two algorithms, the result of the one whose decoding process succeeds earlier is output, so if the decoding process succeeds alternately with the two algorithms, the value of the matching level may become unstable. . In addition, since the image can be easily read by executing the image restoration, the matching level is likely to be higher than the existing one in the matching level calculation method similar to the conventional one. For these reasons, in the present embodiment, the weighted sum of the matching levels of the two algorithms is calculated, and the value of the matching level is adjusted. A specific example will be described below based on the flowchart shown in FIG.

In step SF1 after the start, the read image is input to the decode processing section 210. FIG. In step SF2, the decoding processing unit 210 executes the first decoding process without restoring the image. At step SF3, a matching level A (MLA) is calculated by the first decoding process. On the other hand, in step SF4, the read image is input to the image restoration section 20j to execute image restoration processing. After that, the process proceeds to step SF5, and the decode processing unit 210 executes the second decoding process on the restored image. At step SF6, a matching level B (MLB) is calculated by the second decoding process. If the decoding is not successful in step SF2, MLA=0, and if the decoding is not successful in step SF5, MLB=0.

In step SF7, the values of MLA and MLB are adjusted using, for example, an averaging function. Further, even if the decoding is successful in either step SF2 or SF5, a process of weighting the value of MLA is performed.

(Example of user interface)
FIG. 25 is a diagram showing an example of a user interface 300 displayed on the display section 6. As shown in FIG. The processor 20 generates the user interface 300 and causes the display unit 6 to display it. The user interface 300 includes a first image display area 301 that displays an image (live view image) currently captured by the camera 5, a second image display area 302 that displays a read image, and an image restoration unit 20j. A third display area 303 is provided in which the restored image restored by is displayed. Accordingly, the read image and the restored image can be presented to the user in a form that enables comparison. Also, the user can grasp what kind of image the image on which the decoding process is executed or the image after the decoding process is executed.

(Example of image sensor with AI chip)
FIG. 26 is a diagram showing an example using an AI chip-equipped imaging element 5a, and the imaging element 5a of the camera 5 can be configured as shown in each figure. FIG. 26A is a diagram showing a case in which an AI chip that performs image restoration is provided in the imaging element 5a and packaged, and the read image is output to the processor 20 after image restoration. Also, FIG. 26B is a diagram showing a case in which an AI chip for executing restoration code area extraction and image restoration is provided in the imaging device 5a and packaged. , the partial image corresponding to the region is output to the processor 20 after image restoration. Also, FIG. 26C is a diagram showing a case in which an AI chip for extracting a repair code area is provided in the imaging device 5a and packaged. The partial image to be processed is output to the processor 20 .

Also, FIG. In the example shown in 26A, the read image acquired by the imaging element 5a may be output to the processor 20, and the AI chip may perform image restoration, and the restored image may be output to the processor 20. FIG. FIG. 26B, FIG. The same is true for the example shown in 26C.

(Polarizing filter attachment 3)
In the present embodiment, as shown in FIG. 2, a first illumination section 4a and a second illumination section 4b, which are composed of a plurality of light-emitting diodes, are provided as light sources for generating illumination light for illuminating at least the cord. The polarizing filter attachment 3 includes a first polarizing plate 3a that passes light of a first polarization component out of the light generated by the light emitting diodes that constitute the second lighting unit 4b, and a second polarization plate that is substantially perpendicular to the first polarization component. and a second polarizing plate 3b for passing light of two polarization components. The first polarizing plate 3a and the second polarizing plate 3b can be provided in the hatched areas in FIG. The second polarizing plate 3b is arranged to cover the optical system 5b of the camera 5 from the light incident side.

Therefore, the camera 5 receives the light that passes through the first polarizing plate 3a and is reflected from the surface (code) of the workpiece W via the second polarizing plate 3b. As a result, the regular reflection component of the workpiece W is removed, and the camera 5 generates a read image with a lower contrast than when the first polarizing plate 3a and the second polarizing plate 3b are not interposed. The processor 20 acquires this low-contrast read image. By inputting the low-contrast read image into the neural network, the processor 20 converts it into a restored image with a higher contrast than before the input, and executes decoding processing on the restored image.

That is, by using the polarizing plates 3a and 3b, the specular reflection component of the workpiece W is removed, so that a read image can be acquired with reduced influence of the specular reflection component. For example, the polarizing plates 3a and 3b are suitable for images with many specular reflection components, such as when an image of a metal workpiece is captured, but the read image may become dark due to a decrease in the amount of light, resulting in a decrease in contrast. In such a case, reading performance is improved by converting the image into a high-contrast restored image through neural network inference processing.

The above-described embodiments are merely examples in all respects and should not be construed in a restrictive manner. Furthermore, all modifications and changes within the equivalent range of claims are within the scope of the present invention.

INDUSTRIAL APPLICABILITY As described above, the optical information reader according to the present invention can be used, for example, to read a code attached to a work.

1 stationary optical information reader 1A handheld optical information reader 2 housing 2A handheld housing 2B gripping unit 5 camera 20 processor 20c filter processing unit 20d tuning execution unit 20e setting unit 20f decoding processing unit 20g extraction unit 20h Reduction unit 20i Enlargement unit 20j Image restoration unit 30 Storage unit 30d Neural network storage unit W Work

Claims (17)

An optical information reader that reads a code attached to a work,
a camera that captures the code and produces a read image;
Among a plurality of defective images having portions inappropriate for code reading and a plurality of ideal images respectively corresponding to the plurality of defective images, the defective images in which the pixel resolution of the modules constituting the code is within a specific range. and a storage unit that stores the structure and parameters of a neural network generated in advance by machine learning the ideal image;
The read image generated by the camera and expanded or reduced so that the pixel resolution of the modules constituting the code in the read image is within the specific range is configured with the structure and parameters stored in the storage unit. a processor that attempts to restore the read image according to the structure and parameters stored in the storage unit by inputting the optical information into a neural network stored in the memory, and performs decoding processing on the restored read image. reader. The optical information reader according to claim 1,
The structure and parameters of the neural network are generated in advance by machine learning the defective image and the ideal image in which the number of pixels of the modules constituting the code is within a specific range,
Optical information for inputting the read image generated by the camera to the neural network by enlarging or reducing the number of pixels of the modules constituting the code in the read image generated by the camera to be within the specific range. reader. The optical information reader according to claim 1 or 2,
An optical information reading apparatus further comprising a filter processing unit that performs an image processing filter on the read image generated by the camera before the enlargement or reduction of the read image. In the optical information reader according to any one of claims 1 to 3,
When the optical information reading device is set, a plurality of read images with different enlargement ratios or a plurality of read images with different reduction ratios are generated, and the generated plurality of read images are input to the neural network to obtain each read image. is attempted to be repaired, decode processing is performed on each repaired read image to obtain the read margin, and the read image enlargement or reduction ratio with which the read margin is higher than the specified value is used during operation. further comprising a tuning executor specifying the scaling factor or scale factor to be used;
The processor enlarges or reduces the read image to the enlargement ratio or reduction ratio specified by the tuning execution unit when the optical information reader is operated. In the optical information reader according to claim 4,
The optical information reading device, wherein the tuning executing unit specifies the enlargement ratio or reduction ratio of the read image with the highest read margin among the read margins of the read images as the enlargement ratio or reduction ratio to be used during operation. In the optical information reader according to any one of claims 1 to 3,
The storage unit associates and stores structures and parameters of a plurality of neural networks having different numbers of layers or different numbers of image processing filters with code conditions,
Code conditions included in the read image generated by the camera are set when the optical information reading device is set, and the structure and parameters of the neural network associated with the set code conditions are read from the storage unit and used during operation. further comprising a tuning executor identifying the neural network to be used;
The optical information reading device, wherein the processor attempts restoration of the read image using the neural network specified by the tuning execution unit when the optical information reading device is operated. In the optical information reader according to any one of claims 4 to 6,
The tuning execution unit sets code conditions included in the read image generated by the camera, acquires the number of pixels of the module that is the set code conditions, and if the acquired number of pixels is within the specific range, is an optical information reader that disables said enlargement or reduction by said processor. In the optical information reader according to claim 4,
The storage unit is configured to store a plurality of parameter sets with different enlargement ratios by the processor or a plurality of parameter sets with different reduction ratios by the processor,
The optical information reading device, wherein the processor applies an arbitrary one parameter set from a plurality of parameter sets stored in the storage unit to perform decoding processing. The optical information reader according to any one of claims 1 to 6,
The processor searches for a code in the enlarged or reduced read image based on information for identifying the code, and designates an area containing the searched code as a code candidate area in which the code is likely to exist. extracting and inputting a partial image corresponding to the extracted code candidate region to a neural network configured with the structure and parameters stored in the storage unit, thereby extracting the partial image according to the structure and parameters stored in the storage unit; and decoding the restored partial image. The optical information reader according to any one of claims 1 to 9,
an image restoration unit that inputs the read image generated by the camera to a neural network configured with the structure and parameters stored in the storage unit, and executes inference processing for generating a restored image obtained by restoring the read image;
a first decoding processing unit that executes a first decoding process on the read image generated by the camera;
a second decoding processing unit that executes a second decoding process on the restored image generated by the image restoration unit;
The second decoding processing unit extracts a code candidate area in which a code is likely to exist from the read image, and sends a trigger signal to the image restoration unit to execute inference processing,
When receiving the trigger signal sent from the second decoding processing unit, the image restoration unit inputs a partial image corresponding to the code candidate region extracted by the second decoding processing unit to the neural network, and restores the partial image to the neural network. Execute an inference process that generates a repaired image that is a repaired image,
The second decoding processing unit determines a grid position indicating each cell position of the code based on the restored image, and executes decoding processing of the restored image based on the determined grid position. In the optical information reader according to claim 10,
The inference processing by the image restoration unit and the first decoding processing by the first decoding processing unit are executed in parallel, and at least part of the period during which the second decoding processing unit is executing the second decoding processing is the image restoration. An optical information reader having a rest period during which inference processing by the unit is not performed. The optical information reader according to any one of claims 1 to 11,
an image restoration unit that inputs the read image generated by the camera to a neural network configured with the structure and parameters stored in the storage unit, and executes inference processing for generating a restored image obtained by restoring the read image;
a first decoding processing unit that executes a first decoding process on the read image generated by the camera;
a second decoding processing unit that performs a second decoding process on the restored image generated by the image restoration unit;
a setting unit that sets whether to execute the second decoding process,
The first decoding processing unit is capable of executing the first decoding processing in a time shorter than the time required for the second decoding processing,
When the setting unit is set to execute the second decoding process, the setting unit sets a decoding timeout period longer than the time required to execute the second decoding process, and executes the second decoding process. An optical information reader configured to be able to set a decoding time-out period shorter than the time required for executing the second decoding process when it is set not to do so. The optical information reader according to any one of claims 1 to 9,
an image restoration unit that inputs the read image generated by the camera to a neural network configured with the structure and parameters stored in the storage unit, and executes inference processing for generating a restored image obtained by restoring the read image;
a first decoding process on the read image generated by the camera; and a decoding processing unit that, in parallel with the first decoding process, performs a second decoding process on the restored image generated by the image restoration unit;
a setting unit that sets whether to execute the second decoding process,
When the setting unit sets not to execute the second decoding processing, the decoding processing unit allocates processing resources used for the second decoding processing to the first decoding processing, and performs the second decoding processing. An optical information reader that speeds up the first decoding process compared to when the process is set to be executed. The optical information reader according to any one of claims 1 to 9,
an image restoration unit that inputs the read image generated by the camera to a neural network configured with the structure and parameters stored in the storage unit, and executes inference processing for generating a restored image obtained by restoring the read image;
a decoding processing unit that performs decoding processing on the restored image generated by the image restoration unit;
The decoding processing unit extracts a code candidate area in which a code is likely to exist from the read image,
The image restoration unit inputs a partial image corresponding to the code candidate region extracted by the decoding processing unit to the neural network, and performs an inference process for generating a restored image obtained by restoring the partial image,
The decoding processing unit determines a grid position indicating each cell position of a code based on the restored image, and decodes the restored image based on the determined grid position. The optical information reader according to any one of claims 1 to 9,
an image restoration unit that inputs the read image generated by the camera to a neural network configured with the structure and parameters stored in the storage unit, and executes inference processing for generating a restored image obtained by restoring the read image;
a decoding processing unit that performs decoding processing on the restored image generated by the image restoration unit;
Repeating the imaging and decoding processes by changing the imaging conditions of the camera and the decoding conditions of the decoding process, and based on the matching level indicating the readability of the code calculated under each imaging condition and decoding condition, a tuning execution unit that determines optimum imaging conditions and decoding conditions and executes tuning processing for setting the size of a code to be read;
The decoding processing unit
A code candidate area having a size capable of containing the code to be read set by the tuning process and having a high possibility that the code exists in the read image is extracted, and the extracted image is subjected to a predetermined An optical information reading device that reduces or enlarges the size of the image, inputs it to the neural network, and decodes the restored image that has been restored by the neural network. The optical information reader according to any one of claims 1 to 9,
a plurality of light sources that generate illumination light that illuminates at least the code;
a first polarizing plate for passing light of a first polarization component out of the light generated by the light source;
a second polarizing plate that passes light of a second polarization component substantially orthogonal to the first polarization component,
The camera receives light that has passed through the first polarizing plate and is reflected from the code via the second polarizing plate, and removes specular reflection components of the work, thereby obtaining the first polarizing plate and the second polarizing plate. Generating a low-contrast read image compared to the case where two polarizing plates are not interposed,
The optical information reading device, wherein the processor inputs the low-contrast read image to the neural network to convert it into a restored image with a higher contrast than before the input, and executes decoding processing on the restored image. The optical information reader according to any one of claims 1 to 9,
an image restoration unit that inputs the read image generated by the camera to a neural network configured with the structure and parameters stored in the storage unit, and executes inference processing for generating a restored image obtained by restoring the read image;
a decoding processing unit that performs decoding processing on the restored image generated by the image restoration unit;
The imaging conditions of the camera are changed to repeat imaging and decoding processing, and the imaging conditions and the code of the code to be read are calculated based on the matching level indicating the readability of the code calculated under each imaging condition. a tuning execution unit capable of setting a plurality of conditions,
The storage unit stores a plurality of imaging conditions and code conditions including a first imaging condition and code condition and a second imaging condition and code condition set by the tuning execution unit,
The decoding processing unit inputs to the neural network a read image generated under the first imaging condition and the code condition among the plurality of reading conditions stored in the storage unit to generate a restored image, optical information for generating a restored image by inputting the read image generated under the second imaging condition and the code condition into the neural network when decoding of the restored image fails, and executing decoding of the restored image; reader.

Images