JP2023035594A - Collation apparatus and collation method - Google Patents

Abstract

To provide a collation apparatus configured to check, in real time, whether a component indicated by indication information is moving on a conveyance line while suppressing processing load, and a collation method.SOLUTION: A collation apparatus includes: a receiving unit which receives a signal indicating that a component has been conveyed to a predetermined position; a first acquisition unit which starts, on receipt of the signal, acquiring a first captured image captured by a first imaging apparatus, for an indication medium of indication information that identifies the component; a second acquisition unit which starts, on receipt of the signal, acquiring a second captured image captured by a second imaging apparatus, for a predetermined target in the component; an identifying unit which acquires first information, as output, corresponding to the indication information through identification processing using a learning model to which an image of the indication information included in the first captured image is input, and second information, as output, identifying the target through identification processing using a learning model to which an image of the target included in the second captured image is input; and a collation unit which collates the first information with the second information.SELECTED DRAWING: Figure 3

Description

Applied for application of Article 30, Paragraph 2 of the Patent Act Date of publication August 20, 2021 Address of publication http://gijutsu12/Giken/index. html

The present invention relates to a collation device and a collation method.

In the conventional manufacturing line, the occurrence of erroneous shortages is suppressed by visual inspection by workers to check whether parts such as vehicle bodies are flowing according to the identification information indicated by work instruction boards and the like. In image inspection of parts that are continuously produced at a predetermined repetition pitch, as a technology in such a manufacturing line, the matching processing unit recognizes the execution state of the image inspection. A part is detected from a frame, the position of the detected part and a correlation value are obtained, and the inter-frame part correspondence detection unit detects the correspondence of similar parts between two consecutive frames, and calculates the corresponding part position difference. The unit calculates a corresponding component position difference, which is a difference in detected positions between corresponding components. The corresponding part positional difference statistic part creates a movement amount histogram statistically representing the corresponding part positional difference, and the part movement amount estimating part estimates the movement amount of the part between consecutive frames from the movement amount histogram, and performs the image inspection. A technology is disclosed in which the state determination unit determines whether or not the image inspection is being performed appropriately based on the estimated movement amount of the component (see, for example, Patent Document 1).

Japanese Patent No. 5464092

However, with conventional manufacturing line technology, in order to continuously perform image inspection at a predetermined repetition pitch, the load for image processing and judgment processing, etc. is high. is required.

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems. The purpose is to provide a matching method.

In order to solve the above-described problems and achieve the object, a collating device according to the present invention includes a receiving section for receiving a signal indicating that a part has been conveyed to a predetermined position on a conveying route; a first acquisition unit that, when a signal is received, starts acquisition of a first captured image on a display medium of instruction information for identifying the component, captured by a first imaging device; a second acquiring unit that, when received, starts acquiring a second captured image of a predetermined target object in the component, captured by a second imaging device; and the instruction information included in the first captured image. An image is used as an input, first information corresponding to the instruction information is obtained as an output by a discrimination process using a predetermined learning model, and an image of the object included in the second captured image is used as an input, and the learning model is obtained. It is characterized by comprising an identification unit that obtains as an output second information identifying the object by the identification processing used, and a collation unit that collates the first information and the second information.

ADVANTAGE OF THE INVENTION According to this invention, it can collate in real time whether the components which instruction|indication information shows flow to a conveyance line, suppressing a processing load.

FIG. 1 is a diagram showing an example of the overall configuration of a collation system according to an embodiment. FIG. 2 is a diagram illustrating an example of a hardware configuration of a matching device according to the embodiment; FIG. 3 is a diagram illustrating an example of a functional block configuration of the matching device according to the embodiment; FIG. 4 is a diagram showing an example of a captured image showing instruction information. FIG. 5 is a diagram showing an example of a captured image of an emblem. FIG. 6 is a diagram showing a display example of a captured image on which identification results are superimposed and displayed in the matching system according to the embodiment. FIG. 7 is a flowchart showing an example of the flow of identification matching processing in the matching system according to the embodiment. FIG. 8 is a diagram for explaining the operation of identifying a garnish in a matching device according to a modification.

Embodiments of a collation device and a collation method according to the present invention will be described in detail below with reference to FIGS. 1 to 8. FIG. In addition, the present invention is not limited by the following embodiments, and the constituent elements in the following embodiments can be easily conceived by those skilled in the art, substantially the same, and so-called equivalent ranges. is included. Furthermore, various omissions, replacements, changes and combinations of components can be made without departing from the gist of the following embodiments.

(Overall configuration of matching system)
FIG. 1 is a diagram showing an example of the overall configuration of a collation system according to an embodiment. The overall configuration of a collation system 1 according to this embodiment will be described with reference to FIG.

The verification system 1 shown in FIG. 1 determines whether or not the part indicated by the instruction information is being conveyed from the instruction information indicated by the image captured by the imaging device and the predetermined object of the part on the conveying line 3 (conveyance route). It is a system for collating whether As shown in FIG. 1, the verification system 1 includes a limiter switch 11, a first imaging device 12a, a second imaging device 12b, a verification device 10, a display device 13, and a server device 20.

The limiter switch 11 is a switch for detecting that a part 30 such as a vehicle body flowing in the direction of the arrow shown in FIG. 1 on the transport line 3 has reached a predetermined position. When the limiter switch 11 detects that the part 30 has reached a predetermined position, the limiter switch 11 transmits a detection signal for the part 30 to the matching device 10 . The sensor or switch for detecting that the component 30 has reached a predetermined position on the transport line 3 is not limited to the limiter switch 11. For example, the limiter switch 11 can be replaced by a proximity sensor or a photoelectric A sensor or the like may be used for detection.

The first imaging device 12 a is an imaging device that captures an image of a display medium of instruction information for identifying a component 30 such as a vehicle body flowing on the transport line 3 . The first imaging device 12 a transmits the captured image to the verification device 10 . In this embodiment, the part 30 is the vehicle body of an automobile, the predetermined object is an automobile manufacturer, or an emblem specifying the automobile manufacturer and the vehicle type, and the display medium of the instruction information is shown in FIG. 1, an electric bulletin board 31 (an example of an indication panel) that displays indication information represented by numbers, alphabets, or the like will be described. Note that the component 30, the predetermined target object, and the display medium for the instruction information are not limited to these.

The second image capturing device 12b is an image capturing device that captures an image of a region of the component 30 that includes an emblem, which is a predetermined target object. The second imaging device 12 b transmits the captured image to the matching device 10 .

The images captured by the first imaging device 12a and the second imaging device 12b may be still image data or moving image data.

The verification device 10 converts the instruction information included in the captured image captured by the first imaging device 12a and the emblem included in the captured image captured by the second imaging device 12b into character information. It is a device that verifies The collation device 10 is realized by, for example, a single board computer. The matching device 10 is not limited to being a single board computer, and may be an information processing device such as a normal PC (Personal Computer) or workstation. The matching device 10 includes an image processing program 101 , a learning model 102 and a matching program 103 .

The image processing program 101 is a program for performing image processing such as extracting instruction information and an emblem image from images captured by the first imaging device 12a and the second imaging device 12b.

The learning model 102 includes teacher data (first teacher data) in which predetermined character information as a label is combined with an image of instruction information, and teacher data (first teacher data) in which an image of an emblem is combined with predetermined character information as a label. 2 teacher data) is used to generate a learning model by supervised learning, which is an example of machine learning. The learning model may be generated based on an existing object recognition algorithm such as YOLO (You Only Look Once). Here, the predetermined character information is assumed to be character information such as the name of the manufacturer or the name of the vehicle type corresponding to the emblem, for example. In this embodiment, as a result of identification processing using the learning model 102, character information indicating the name of the manufacturer and reliability indicating the degree of reliability of the result are obtained from the image of the instruction information and the image of the emblem. described as a thing.

The collation program 103 is a program for collating the character information for the instruction information image and the character information for the emblem image obtained as a result of the identification processing using the learning model 102 .

The display device 13 is a display device such as an LCD (Liquid Crystal Display) or an OELD (Organic Electro-Luminescent Display) that displays the collation results of the collation program 103 .

The server device 20 is a server device that accumulates matching results and the like by the matching program 103 .

(Hardware configuration of verification device)
FIG. 2 is a diagram illustrating an example of a hardware configuration of a matching device according to the embodiment; The hardware configuration of the matching device 10 according to this embodiment will be described with reference to FIG.

As shown in FIG. 2, the verification device 10 includes a CPU (Central Processing Unit) 201, a RAM (Random Access Memory) 202, a ROM (Read Only Memory) 203, an auxiliary storage device 204, and a network I/F 205. , an input I/F 206 , an output I/F 207 , and an imaging I/F 208 . These units are connected via a bus line 210 so as to enable data communication with each other.

The CPU 201 is an arithmetic device that controls the operation of the collation device 10 as a whole. A RAM 202 is a volatile storage device used as a work area for the CPU 201 . The ROM 203 is a non-volatile storage device that stores programs such as an IPL (Initial Program Loader) that the CPU 201 executes first. Note that the CPU 201, the RAM 202, and the ROM 203 may be configured as an SoC (System on a Chip) mounted on one substrate, for example.

Auxiliary storage device 204 includes HDD (Hard Disk Drive), SSD (Solid State Drive), eMMC ( embedded Multi Media Card) or an auxiliary storage device such as a microSD card.

A network I/F 205 is an interface for connecting to a network or the like for data communication. In the example shown in FIG. 2 , the network I/F 205 is connected to the server device 20 and transmits the above-mentioned matching result and the like in the matching device 10 to the server device 20 . Also, the network I/F 205 is an Ethernet (registered trademark) standard interface that enables communication using TCP (Transmission Control Protocol)/IP (Internet Protocol) protocols, for example. Note that the network I/F 205 is not limited to wired communication, and may be a wireless communication interface based on Wi-Fi (registered trademark) or the like.

The input I/F 206 is an interface such as a GPIO (General Purpose Input/Output) for inputting a detection signal from the limiter switch 11 .

The output I/F 207 is an interface such as HDMI (registered trademark) (High-Definition Multimedia Interface) for transmitting display data such as matching results to the display device 13 .

The imaging I/F 208 is an interface such as a USB (Universal Serial Bus) for receiving captured images from the first imaging device 12a and the second imaging device 12b.

Note that the hardware configuration of the matching device 10 shown in FIG. 1 is an example, and is not limited to this configuration. For example, it may include a GPU (Graphics Processing Unit) that specializes in graphic processing.

(Functional block configuration of collation device)
FIG. 3 is a diagram illustrating an example of a functional block configuration of the matching device according to the embodiment; FIG. 4 is a diagram showing an example of a captured image showing instruction information. FIG. 5 is a diagram showing an example of a captured image of an emblem. FIG. 6 is a diagram showing a display example of a captured image on which identification results are superimposed and displayed in the matching system according to the embodiment. The functional block configuration and operation of the matching device 10 according to the present embodiment will be described with reference to FIGS. 3 to 6. FIG.

As shown in FIG. 3, the verification device 10 includes a first acquisition unit 301, a second acquisition unit 302, a detection unit 303 (reception unit), an extraction unit 304, an identification unit 305, a verification unit 306, It has a display control unit 307 , a generation unit 308 , and a storage unit 309 .

The first acquisition unit 301 is a functional unit that acquires a captured image of the electronic bulletin board 31 captured by the first imaging device 12a (hereinafter sometimes referred to as a first captured image). Here, FIG. 4B shows a captured image IMG2 of the electronic bulletin board 31 captured by the first imaging device 12a as an example of the first captured image. As described above, the target imaged by the first imaging device 12a is not limited to the electronic bulletin board 31 that displays the instruction information, and is, for example, the component 30 as shown in FIG. It may be a captured image IMG1 (an example of a first captured image) including an instruction sheet on which instruction information is printed and attached to the vehicle body. Further, when the first acquisition unit 301 receives from the detection unit 303, which will be described later, a detection signal regarding the component 30, it starts acquiring the first captured image captured by the first imaging device 12a. At the timing when the detection signal is received by the detection unit 303, the imaging operation by the first imaging device 12a and the acquisition operation of the first captured image by the first acquisition unit 301 may be started. The first acquisition unit 301 outputs the acquired first captured image to the extraction unit 304 . The first acquisition unit 301 is implemented by executing the image processing program 101 by the CPU 201 shown in FIG. 2, for example.

The second acquisition unit 302 is a functional unit that acquires a captured image (hereinafter sometimes referred to as a second captured image) including the emblem of the vehicle body that is the part 30 captured by the second imaging device 12b. Here, FIG. 5 shows, as an example of the second captured image, a captured image IMG3 including the emblem of the vehicle body captured by the second imaging device 12b. Further, when the second acquisition unit 302 receives a detection signal regarding the component 30 from the detection unit 303 (to be described later), it starts acquisition of the second captured image captured by the second imaging device 12b. At the timing when the detection signal is received by the detection unit 303, the imaging operation by the second imaging device 12b and the acquisition operation of the second captured image by the second acquisition unit 302 may be started. The second acquisition unit 302 outputs the acquired second captured image to the extraction unit 304 . The second acquisition unit 302 is implemented by executing the image processing program 101 by the CPU 201 shown in FIG. 2, for example.

The detection unit 303 is a functional unit that receives a detection signal from the limiter switch 11 when the limiter switch 11 detects that the component 30 has reached a predetermined position on the conveying line 3 . Upon receiving the detection signal, the detection unit 303 outputs to the first acquisition unit 301 and the second acquisition unit 302 that the detection signal has been received. The detection unit 303 is realized by executing a program such as the image processing program 101 by the CPU 201 shown in FIG. 2, for example.

The extraction unit 304 is a functional unit that extracts a predetermined image portion from the captured images respectively acquired by the first acquisition unit 301 and the second acquisition unit 302 .

Specifically, the extraction unit 304 extracts an image including instruction information for identifying the emblem from the first captured image acquired by the first acquisition unit 301 . For example, in the example shown in FIG. 4B, the extraction unit 304 extracts an extracted image EIMG2 as an image including instruction information from the captured image IMG2, which is the first captured image received from the first acquisition unit 301. FIG. Note that, as shown in FIG. 4A, when the first captured image is a captured image IMG1 including an instruction sheet on which instruction information is printed, the extraction unit 304 extracts the instruction information from the captured image IMG1. Extraction image EIMG1 may be extracted as an image.

Also, the extraction unit 304 extracts an image including the emblem from the second captured image acquired by the second acquisition unit 302 . For example, in the example shown in FIG. 5, the extracting unit 304 extracts an extracted image EIMG3 as an image including the emblem from the captured image IMG3, which is the second captured image received from the second acquiring unit 302. FIG.

As a method for the extraction unit 304 to extract a predetermined image portion (instruction information and emblem) from each captured image, for example, the image area to be extracted may be set in advance, or a predetermined method such as template matching may be used. The algorithm may extract predetermined image portions. The extraction unit 304 outputs the extracted instruction information image and the emblem image to the identification unit 305 . The extracting unit 304 is implemented by executing the image processing program 101 by the CPU 201 shown in FIG. 2, for example.

The identifying unit 305 receives the extracted image received from the extracting unit 304 as an input, performs identification processing using the learning model 102 stored in the storage unit 309, and outputs character information indicating the manufacturer name corresponding to the extracted image. is. Specifically, the identification unit 305 performs identification processing using the learning model 102 based on the image of the instruction information received from the extraction unit 304, and character information (hereinafter referred to as character information) indicating the manufacturer name corresponding to the image of the instruction information. , may be referred to as first character information), and a reliability indicating the degree of reliability of the identification process. Based on the image of the emblem received from the extraction unit 304, the identification unit 305 performs identification processing using the learning model 102 to perform character information (hereinafter referred to as second character information) indicating the manufacturer name corresponding to the image of the emblem. ), and a reliability indicating the degree of reliability of the identification process. Identification unit 305 outputs the first character information, the second character information, and their degrees of reliability to collation unit 306 . The identification unit 305 is implemented by, for example, executing a program for performing identification processing using the learning model 102 by the CPU 201 shown in FIG.

Although the first character information has been described as the manufacturer's name, it is not limited to this, and any character information corresponding to the instruction information may be used. Also, although the second character information has been described as the manufacturer's name, the second character information is not limited to this, and may be any character information that identifies a predetermined object such as an emblem. Also, the first character information and the second character information are not limited to character information, and may be, for example, binary data.

The collation unit 306 is a functional unit that collates whether or not the first character information (an example of the first information) received from the identification unit 305 matches the second character information (an example of the second information). Collation unit 306 outputs the collation result to display control unit 307 and server device 20 . Collation unit 306 also outputs reliability levels corresponding to the first character information and the second character information to display control unit 307 and server device 20 . The collation unit 306 is implemented by executing the collation program 103 by the CPU 201 shown in FIG. 2, for example.

Note that the collation unit 306 may transmit the collation result to a host computer that controls the transport line 3 . As a result, the high-level computer can execute appropriate processing such as stopping the transport line 3 when the collation result is inconsistent.

The display control unit 307 is a functional unit that causes the display device 13 to display the matching results received from the matching unit 306 . For example, the display control unit 307 superimposes the matching result and the second character information corresponding to the image of the emblem on the second captured image acquired by the second acquisition unit 302 and causes the display device 13 to display the superimposed image. The reliability corresponding to the second character information may be superimposed on the second captured image instead of or in addition to the second character information.

In the example shown in FIG. 6A, the display control unit 307 superimposes the result display unit RST, which is the matching result, on the captured image IMG4, which is the second captured image acquired by the second acquisition unit 302. The superimposed display part SPN1 including the second character information "Manufacturer A" and the reliability "0.82" is displayed on the image part of the emblem, and the part surrounding the image part of the emblem is superimposed and displayed on the display device 13. I am letting In FIG. 6(b), a superimposed display portion SPN2 including a portion surrounding the image portion of the emblem, including the second character information "Manufacturer B" and the reliability "0.80", is superimposed on the image portion of the emblem. example. In FIG. 6(c), a superimposed display portion SPN3 including the second character information "Manufacturer C" and the reliability "0.96" and surrounding the image portion of the emblem is superimposed on the image portion of the emblem. example. In this case, the display control section 307 may display the superimposed display sections SPN1 to SPN3 only when the matching result indicates a match (when "OK"). In addition, as shown in FIG. 6, the display control unit 307 may display the superimposed display units SPN1 to SPN3 in different modes such as display colors in accordance with, for example, the manufacturer name indicated by the second character information. .

By displaying the collation result on the display device 13 in this way, the operator of the transfer line 3 can visually recognize the collation result. Also, by displaying the second character information superimposed on the emblem image portion of the second captured image, the worker can visually confirm the manufacturer name corresponding to the emblem identified by the learning model 102. be able to. Also, by superimposing the reliability of the identification processing using the learning model 102 on the emblem image portion of the second captured image, the reliability of the identification processing using the learning model 102 can be confirmed. Also, by superimposing and displaying the display portion surrounding the image portion of the emblem in the second captured image, it is possible to easily recognize the emblem, which is the object of identification processing by the learning model 102, on the image.

The display control unit 307 is implemented, for example, by executing a program such as the matching program 103 by the CPU 201 shown in FIG.

The generation unit 308 generates training data in which the image of the instruction information is combined with the manufacturer name corresponding to the instruction information as a label, and training data in which the manufacturer name corresponding to the emblem is combined with the image of the emblem as the label. It is a functional unit that generates a learning model 102 in advance by using supervised learning, which is an example of machine learning. The generation unit 308 causes the storage unit 309 to store the generated learning model 102 . The generation unit 308 is realized, for example, by the CPU 201 shown in FIG. 2 executing a learning program that realizes supervised learning.

The storage unit 309 is a functional unit that stores the image processing program 101, the learning model 102, the matching program 103, and various data used in these programs. As shown in FIG. 1 above, the image processing program 101, the program using the learning model 102, and the matching program 103 are separate programs. It may be configured as one program. The storage unit 309 is implemented by the auxiliary storage device 204 shown in FIG.

Note that each functional unit of the collation device 10 shown in FIG. 3 conceptually shows the function, and is not limited to such a configuration. For example, a plurality of functional units illustrated as independent functional units in FIG. 3 may be configured as one functional unit. On the other hand, the function of one functional unit in FIG. 3 may be divided into a plurality of functions to form a plurality of functional units.

(Flow of identification matching process of matching system)
FIG. 7 is a diagram showing an example of the flow of identification matching processing in the matching system according to the embodiment. The flow of identification matching processing of the matching system 1 according to this embodiment will be described with reference to FIG.

<Step S11>
First, when the limiter switch 11 detects the part 30, which is a vehicle body, flowing on the conveying line 3, and the detection unit 303 receives a detection signal from the limiter switch 11 (step S11: Yes), the verification device 10 shifts from the standby mode to the normal state. , the detection unit 303 outputs the reception of the detection signal to the first acquisition unit 301 and the second acquisition unit 302, and proceeds to step S12. If the component 30 is not detected by the limiter switch 11 (step S11: No), it waits until it is detected. Here, the standby mode is, for example, an operation mode in a power saving state.

<Step S12>
When receiving the detection signal from the detection unit 303, the first acquisition unit 301 starts acquiring the first captured image captured by the first imaging device 12a, and extracts the acquired first captured image. Output to unit 304 . Further, when the second acquisition unit 302 receives the reception of the detection signal from the detection unit 303, the second acquisition unit 302 starts acquisition of the second captured image captured by the second imaging device 12b, and acquires the acquired second captured image. , to the extraction unit 304 . Then, the process proceeds to step S13.

<Step S13>
The extraction unit 304 extracts an image including instruction information for identifying the emblem from the first captured image acquired by the first acquisition unit 301 and outputs the image to the identification unit 305 . The extraction unit 304 also extracts an image including the emblem from the second captured image acquired by the second acquisition unit 302 and outputs the image to the identification unit 305 . Then, the process proceeds to step S14.

<Step S14>
Based on the image of the instruction information received from the extraction unit 304, the identification unit 305 performs identification processing using the learning model 102, and performs first character information indicating the manufacturer name corresponding to the image of the instruction information and the identification processing information. Output a confidence level that indicates the degree of confidence. Based on the image of the emblem received from the extraction unit 304, the identification unit 305 performs identification processing using the learning model 102, and performs second character information indicating the name of the manufacturer corresponding to the image of the emblem and identification processing information. Output a confidence level that indicates the degree of confidence. Identification unit 305 then outputs the first character information, the second character information, and the reliability of each to collation unit 306 . Then, the process proceeds to step S15.

<Step S15>
The collation unit 306 collates whether or not the first character information and the second character information received from the identification unit 305 match. Collation unit 306 outputs the collation result to display control unit 307 and server device 20 . Collation unit 306 also outputs reliability levels corresponding to the first character information and the second character information to display control unit 307 and server device 20 . Then, the process proceeds to step S16.

<Step S16>
The display control unit 307 superimposes the result display portion RST, which is the matching result, on the second captured image acquired by the second acquisition unit 302, and displays the second character information and its reliability on the image portion of the emblem. A superimposed display portion including a display and surrounding the image portion of the emblem is superimposed and displayed on the display device 13 . When the matching process by the matching unit 306 and the display process by the display control unit 307 are finished, the matching device 10 shifts to the standby mode. Then, the identification matching process ends.

As described above, in the matching device 10 according to the present embodiment, the detection unit 303 receives a detection signal indicating that the component 30 has been transported to a predetermined position on the transport line 3, and the first acquisition unit 301 When the detection signal is received by the detection unit 303, acquisition of the first captured image for the display medium of the instruction information for identifying the component 30, which is captured by the first imaging device 12a, is started, and the second acquisition unit 302 , when the detection signal is received by the detection unit 303, acquisition of a second captured image of the emblem on the part 30 captured by the second imaging device 12b is started, and the identification unit 305 detects the image included in the first captured image. The first character information is obtained from the image of the instruction information by performing identification processing using the learning model 102, and the second character information is obtained from the image of the emblem included in the second captured image by identification processing using the learning model 102. , a collating unit 306 collates the first character information and the second character information. As a result, a series of processes from the time when the detection signal is received to the start of the captured image acquisition process to the collation process are executed. can be collated in real time while suppressing the processing load.

In the matching device 10, the extraction unit 304 extracts the image of the instruction information from the first captured image, and extracts the image of the emblem from the second captured image. As a result, the accuracy of identification processing by the learning model 102 can be improved.

Further, in the collation device 10, the display medium is an indication board such as an electronic bulletin board 31 installed on the transport line 3 to display indication information. This makes it easier to extract the image portion of the instruction information from the first captured image, so that the instruction information can be recognized more reliably.

When the detection signal is received by the detection unit 303 , the verification device 10 returns from the standby mode, and when the verification processing by the verification unit 306 ends, the verification device 10 shifts to the standby mode. Thereby, a single board computer or the like can be adopted as the matching device 10, and the processing load can be reduced.

(Modification)
The verification system 1 according to the modified example will be described with a focus on the differences from the verification system 1 according to the above-described embodiment. In the above-described embodiment, the predetermined target object of the part 30 to be imaged is explained as being the emblem. explain.

FIG. 8 is a diagram for explaining the operation of identifying a garnish in a matching device according to a modification. A verification device 10 according to this modification will be described with reference to FIG. 8 .

In this modified example, as shown in FIG. 8, the second image capturing device 12b captures an image of a region including a garnish as a predetermined target in a component 30 that is a vehicle body. In this case, regarding the garnish attached to the component 30, it is important to determine whether the garnish of the correct color is attached. The instruction information is assumed to be information for identifying the color of the garnish as the predetermined object.

In this modification, the learning model 102 uses teacher data in which predetermined text information as a label is combined with an image of instruction information, and teacher data in which a garnish image is combined with predetermined text information as a label. , is generated in advance by the generation unit 308 by supervised learning, which is an example of machine learning. Here, the predetermined character information is assumed to be character information indicating a color, for example.

The extraction unit 304 extracts an image including instruction information for identifying the color of the garnish from the first captured image acquired by the first acquisition unit 301 . Also, the extraction unit 304 extracts an image including the garnish from the second captured image acquired by the second acquisition unit 302 . For example, in the example shown in FIG. 8A, the extraction unit 304 extracts an extracted image EIMG11 as an image including black garnish from the captured image IMG11, which is the second captured image. In the example shown in FIG. 8B, the extraction unit 304 extracts an extracted image EIMG12 as an image including white garnish from the captured image IMG12, which is the second captured image. The extraction unit 304 outputs the extracted instruction information image and the garnish image to the identification unit 305 .

Based on the extraction image received from the extraction unit 304, the identification unit 305 performs identification processing using the learning model 102 stored in the storage unit 309, and outputs character information indicating the color corresponding to the extraction image. Specifically, based on the image of the instruction information received from the extraction unit 304, the identification unit 305 performs identification processing using the learning model 102 to perform first character information indicating a color corresponding to the image of the instruction information, and a reliability indicating the degree of reliability of the identification process. Based on the image of the garnish received from the extraction unit 304, the identification unit 305 performs identification processing using the learning model 102, and performs second character information indicating the color corresponding to the image of the garnish and Output a confidence level that indicates the degree of confidence. Identification unit 305 outputs the first character information, the second character information, and their degrees of reliability to collation unit 306 .

The collation unit 306 is a functional unit that collates whether or not the first character information and the second character information received from the identification unit 305 match. Collation unit 306 outputs the collation result to display control unit 307 and server device 20 . Collation unit 306 also outputs reliability levels corresponding to the first character information and the second character information to display control unit 307 and server device 20 .

Other configurations of the verification system 1 according to this modification are the same as those of the verification system 1 according to the above-described embodiment.

As described above, in the collating device 10 according to the present modification, it is possible to use other members such as garnishes instead of emblems as objects of the part 30. By using these objects, Whether or not the component 30 indicated by the instruction information is flowing on the transfer line 3 can be verified in real time while suppressing the processing load.

1 Verification System 3 Conveyance Line 10 Verification Device 11 Limiter Switch 12a First Imaging Device 12b Second Imaging Device 13 Display Device 20 Server Device 30 Part 31 Electronic Bulletin Board 101 Image Processing Program 102 Learning Model 103 Verification Program 301 First Acquisition Section 302 Second 2 acquisition unit 303 detection unit 304 extraction unit 305 identification unit 306 matching unit 307 display control unit 308 generation unit 309 storage unit

Claims (6)

a receiving unit that receives a signal indicating that the part has been transported to a predetermined position on the transport route;
a first acquisition unit that, when the signal is received by the reception unit, starts acquisition of a first captured image for a display medium of the instruction information for identifying the component, which is captured by the first imaging device;
a second acquisition unit that starts acquisition of a second captured image of a predetermined target object in the component captured by a second imaging device when the signal is received by the reception unit;
The image of the instruction information included in the first captured image is input, the first information corresponding to the instruction information is obtained as an output by identification processing using a predetermined learning model, and the first information included in the second captured image is obtained. an identification unit that receives an image of an object as an input and obtains as an output second information for identifying the object through identification processing using the learning model;
a matching unit for matching the first information and the second information;
Verification device with 2. The matching device according to claim 1, further comprising an extraction unit that extracts the image of the instruction information from the first captured image and extracts the image of the target from the second captured image. 3. The collation device according to claim 2, wherein the display medium is an indication panel installed on the conveying path and displaying the indication information. supervised learning using first training data combining an image and a label of instruction information for identifying the component and second training data combining an image and a label of a predetermined object in the component; 4. The matching device according to any one of claims 1 to 3, further comprising a generator that generates a learning model. The collation device according to any one of claims 1 to 4, wherein when the signal is received by the reception unit, the standby mode is returned, and when the collation processing by the collation unit is completed, the standby mode is entered. a receiving step of receiving a signal indicating that the part has been transported to a predetermined position on the transport path;
a first acquisition step of starting acquisition of a first captured image on a display medium of instruction information for identifying the component, which is captured by a first imaging device when the signal is received;
a second acquisition step of starting acquisition of a second captured image of a predetermined object in the component captured by a second imaging device when the signal is received;
The image of the instruction information included in the first captured image is input, the first information corresponding to the instruction information is obtained as an output by identification processing using a predetermined learning model, and the first information included in the second captured image is obtained. an identification step of obtaining, as an output, an image of an object as an input and second information for identifying the object through an identification process using the learning model;
a matching step of matching the first information and the second information;
A matching method that has

Images