TWI855579B - Inspection system and artificial intelligence model data management method - Google Patents

Abstract

The present invention provides a technology that can more easily apply a suitable AI model to a production line. An inspection system (1) uses an AI model in a production line (100) of a product to perform inspection based on image data of the product. The inspection system (1) includes a management information display unit (10c), the management information display unit (10c) displays information related to the management of one or more AI models used for product inspection, and the management information display unit (10c) can display specification information related to the specifications of the AI model and performance information related to the performance of the AI model for various data, the performance information including the performance of the AI model for model testing before mass production application and the performance when applied to mass production.

Description

Inspection system and artificial intelligence model data management method

The present invention relates to an inspection system for inspecting the quality of products during the production of products and a management method for artificial intelligence (AI) model data.

In the production line of finished products, product inspection equipment is configured in the middle step or the final step of the production line to detect defects or classify defective products. For example, in the production line of component mounting substrates, it is usually included that there is a device for printing paste solder on the printed wiring substrate (printing device), a device for mounting components on the substrate printed with paste solder (assembly device), and a device for heating the substrate after component mounting and soldering the components to the substrate (reflow device). In addition, in the inspection system including the inspection equipment configured after each production device, it is checked whether the operation in each production device is carried out accurately as planned. Then, in the inspection system, the information of each inspection result is collected and managed, so that the change of defective rate can be appropriately and quickly responded, which helps to improve the productivity of the entire production line.

In the inspection system described above, by photographing an inspection object such as a product, the image is input to the previously learned AI model, and a step such as inspecting the quality of the product is introduced. Here, the quality of mass-produced products in the production line is constantly changing. If the change increases, the AI model cannot accurately determine the quality of the product, so the risk of over-detection or under-detection increases. In this case, it is necessary to use newly acquired learning data to re-learn the AI model and use the best AI model corresponding to the change in quality.

In this regard, the following technology is known: by making the image judgment model learn new good product images and defective product images, a model that can appropriately judge data with unprecedented new characteristics can be generated (see Patent Document 1). However, the performance of the AI model (= over-reading rate, missed reading rate) will not become smarter if the learning data is simply increased, and if the overall characteristics of the learning data and the test data deviate, there is a risk of over-learning.

Overlearning is considered to occur at various levels, such as the following patterns. (1) A pattern in which high performance is achieved for learning data, but high performance is not achieved for test data, library data, and actual mass production data (2) A pattern in which high performance is achieved for learning data and test data, but high performance is not achieved for library data and actual mass production data (3) A pattern in which high performance is achieved for learning data, test data, and library data, but high performance is not achieved for actual mass production data As described above, due to the divergence between the performance for specific overall data and the performance for a wide range of mass production data, the pre-learned AI model may not perform as expected in mass production. Furthermore, in response to this problem, there is an increasing demand for a system that can apply a more appropriate AI model to the production line during mass production. [Prior art literature] [Patent literature]

[Patent Document 1] Japanese Patent Publication No. 2020-107104

[The problem that the invention wants to solve]

In view of the above-mentioned actual situation, the purpose of the present invention is to provide a technology that can more easily apply a suitable AI model to a production line. [Means for solving the problem]

In order to achieve the above-mentioned purpose, the present invention adopts the following structure. That is, An inspection system, in a production line of a product, uses an AI model to perform inspection based on the image data of the product, The inspection system is characterized by including: A management information display unit, which displays information related to the management of one or more AI models used for the inspection of the product, The management information display unit can display specification information related to the specifications of the AI model, and performance information related to the performance of the AI model for various data, The performance information includes the performance of the AI model for model testing before mass production application, and the performance when applied to mass production.

This allows users to have an overview of the AI model's specifications and performance information, and analyze the cause when, for example, the performance of the AI model in the model test before mass production application deviates from the performance when applied to mass production. In addition, the AI model can be changed to another model or used for learning and evaluation using new image data, making it easier to apply the appropriate AI model to the production line.

Furthermore, in the present invention, the management information display unit can display the performance of the AI model in the model test before mass production application, the performance when applied to mass production, and the status of application to mass production as the performance information on one screen. In this way, the user can quickly decide on the policy of changing and relearning the AI model by only observing the one screen.

Furthermore, in the present invention, the model test before mass production of the AI model may include at least one of a model test on test data or a model test on library data. This allows the performance of deviations and different data to be quickly grasped, thereby also allowing the AI model to be quickly changed, re-learned, and other policies to be decided.

Moreover, in the present invention, the performance may include an over-reading indicator based on the number of good products judged as defective products by the AI model in an inspection without using the AI model, and an under-reading indicator based on the number of defective products (actually defective) judged as good products by the AI model in an inspection without using the AI model. Thereby, it is possible to immediately identify whether the judgment of the AI model is too strict or too loose, and to judge whether the model is over-learned, and the effectiveness of the model. Furthermore, "inspection without using the AI model" may also be a visual inspection. Or it may also be other inspections with high accuracy in judging the quality. For example, when the judgment accuracy is high, it may also be a rule library inspection.

Furthermore, in the present invention, the performance information may further include the model test before mass production application of the AI model and each inspection image during mass production, the AI model's judgment result on the quality of the inspection image, and the AI output value related to the inspection image. In this way, for each inspection image, the AI model can confirm what kind of judgment has been made, and can more appropriately determine the policy of changing and relearning the AI model.

Moreover, in the present invention, the performance information may further include a histogram of the AI output value of the AI model with respect to the inspection image. This makes it easier to confirm the tendency of the AI model to judge the inspection image.

Furthermore, in the present invention, the management information display unit can display the performance information for each inspection item of the product. This allows the performance of the AI model for each inspection item to be confirmed, and allows the policy of changing and relearning the AI model to be determined in more detail.

Furthermore, in the present invention, the management information display unit can switch the display of the specification information and the performance information by displaying each model group of the inspection target parts and displaying a summary of multiple model groups. In this way, the performance of the AI model of each model group can be confirmed in detail, and the relocation or replacement of AI models between different model groups can be studied more easily. Here, the so-called model group refers to a collection of models with similar part sizes or part colors. The models in the model group are inspected with the same inspection criteria and color parameters, so there is no need to teach with each model, which can improve the efficiency of teaching.

Furthermore, in the present invention, the management information display unit can execute a process of applying the AI model applied to one model group to other model groups in a display screen that summarizes a plurality of model groups. Thus, the user can immediately apply the AI model applied to one model group to other model groups while observing the display screen of the management information display unit.

Furthermore, in the present invention, the specification information may include at least one of network information of the AI model, information of learning materials during learning, and a lot number of the AI model. Thus, the user can more easily confirm the basic information and traceability of the AI model.

Moreover, the present invention may be a method for managing AI model data, which is used in an inspection system that uses an AI model in a production line of a product to perform inspections based on image data of the product. The method for managing AI model data is characterized in that: The AI model data is managed by associating specification information related to the specifications of the AI model data and performance information related to the performance of the AI model for various data. The performance information includes the performance of the AI model for model testing before mass production and the performance when applied to mass production. Furthermore, here, the so-called management includes not only memorizing and saving information, but also, for example, real-time inspection based on image data to display performance information for the image data.

Furthermore, the present invention may be a method for managing the AI model data, characterized in that the model test before the mass production application of the AI model includes at least one of a model test on test data or a model test on library data.

Moreover, the present invention can be a method for managing the AI model data, which is characterized in that: the performance includes an over-reading indicator based on the number of good products judged as defective products in an inspection without using the AI model by the AI model, and a missing indicator based on the number of defective products judged as good products in an inspection without using the AI model by the AI model.

Moreover, the present invention can be a method for managing the AI model data, which is characterized in that the performance information further includes model testing before mass production application of the AI model and various inspection images during mass production, the judgment results of the AI model on the quality of the inspection image, and the AI output value related to the inspection image.

Furthermore, the present invention may be a method for managing the AI model data, characterized in that the specification information includes at least any one of network information of the AI model, information of learning data during learning, and a batch number of the AI model.

Moreover, the present invention may be an AI model data set, including one or more AI model data, for use in an inspection system that uses an AI model in a production line of a product to perform inspection based on image data of the product, wherein the AI model data set is characterized in that: It includes specification information related to the specifications of the AI model data and performance information related to the performance of the AI model for various data, The performance information includes each inspection image and the judgment result of the AI model on the quality of the inspection image.

Moreover, the present invention may be the AI model dataset, characterized in that: the performance includes an over-reading indicator based on the number of good products judged as defective products in an inspection without using the AI model by the AI model, and a missing indicator based on the number of defective products judged as good products in an inspection without using the AI model by the AI model.

Moreover, the present invention may be the AI model dataset, characterized in that the performance information further includes the AI output value related to the inspection image.

Furthermore, the present invention may be the AI model dataset, characterized in that the specification information includes at least any one of network information of the AI model, information of learning data during learning, and a batch number of the AI model.

Furthermore, as long as the above structures and processes do not cause technical contradictions, they can be combined with each other to constitute the present invention. [Effect of the invention]

According to the present invention, suitable AI models can be more easily applied to production lines.

<Application example> As shown in FIG. 1, a production line 100 as an example of an object to which the present invention is applied includes a solder printing device 100a, a post-solder printing inspection device 100b, an assembly machine 100c, a post-assembly inspection device 100d, a reflow furnace 100e, and a post-reflow inspection device 100f. Each device in the production line 100 is connected to a teaching terminal 10 via a network such as a local area network (LAN). Each inspection device 100b, inspection device 100d, and inspection device 100f inspects the state of the printed circuit board at the exit of each step and automatically detects defects or the possibility of defects.

Furthermore, the inspection in each inspection device of the production line 100 is performed using a pre-learned AI model. The AI model is learned in advance using learning data using the teaching terminal 10. Thereafter, the performance is evaluated based on test data, library data, and mass production data.

Here, the AI model is learned using data with small deviations and period-related data of the learning data, so it may not be able to fully demonstrate its performance when inspecting mass production data with large deviations and period-related data. In response to this, in this application, for multiple AI models, users can view the specification information when the AI model was generated and the performance information of what kind of performance is demonstrated for test data, library data, and mass production data, and can more easily apply the best AI model to mass production.

FIG8 shows an example of an AI model management screen 21 in this application case. An AI usage history display area 22 is provided in the AI model management screen 21. The AI usage history display area 22 is an area for displaying the usage history of each AI model in mass production. The model name of the AI model used in mass production is recorded at the left end 22b of the AI usage history display area 22. The period of use of the model is displayed in a bar graph in the column recording each model name. The AI usage history display area 22 can display historical information of the period of use of each model in mass production.

Furthermore, the AI model management screen 21 includes a performance display area 23 on the right side of the AI use history display area 22. For each AI model, the performance for the test data is displayed in the left column 23a of the performance display area 23. Moreover, for each AI model, the performance for the library data is displayed in the right column 23b of the performance display area 23. More specifically, the over-view rate and the missed rate when the test data is used, and the over-view rate and the missed rate when the library data is used are displayed in a bar graph.

Furthermore, a mass production performance display area 24 is provided below the AI usage history display area 22. The mass production performance display area 24 displays the performance of the AI model used on each working day shown in the AI usage history display area 22 for the mass production data. More specifically, the over-view rate and the missed-view rate are displayed in a bar graph.

Moreover, a detailed display button 21d is provided at the right end of the AI model management screen 21 corresponding to the model name of each AI model. By pressing the detailed display button 21d, detailed information of each AI model is displayed. FIG9 shows an example of a first detailed display screen 31a displayed by pressing the detailed display button 21d. A display mode selection area 32 is displayed on the first detailed display screen 31a, and in the display mode selection area 32, the display content in the detailed display screen can be selected. In FIG9, basic information is selected in the display mode selection area 32. Thereby, the first detailed display screen 31a becomes a screen for displaying basic information.

As shown in FIG9 , a basic information display area 33 is provided in the first detailed display screen 31a. In the basic information display area 33, the model name of the target AI model, the date and time of generation, the image size of the image to be inspected, the network name used, the name of the learning data, the collection period of the learning data, the lot number of the product to be inspected, and comments are recorded.

In this application, as described above, users can easily check the basic information (specification information) of each AI model and the performance information of each test and mass production at a glance. As a result, it is easier or more accurate to make decisions such as changing the AI model used in mass production or relearning it.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, unless otherwise specified, the scope of the present invention is not limited to the structural elements described in the following examples.

<Example> A production line 100 as an example of an object of the invention to which the present invention is applied is shown in FIG1. As described above, the production line 100 includes, for example, a solder printing device 100a, a post-solder printing inspection device 100b, an assembly machine 100c, a post-assembly inspection device 100d, a reflow furnace 100e, and a post-reflow inspection device 100f. The solder printing device 100a is a device for printing solder paste on an electrode portion on a printed substrate. The assembly machine 100c is a device for placing a large number of electronic components to be mounted on a printed substrate on the solder paste. Furthermore, the reflow furnace 100e is a heating device for soldering the electronic components placed on the printed substrate to the printed wiring on the substrate. Furthermore, each inspection device 100b, inspection device 100d, and inspection device 100f inspects the state of the printed circuit board at the exit of each step and automatically detects defects or the risk of defects.

The above-mentioned post-solder printing inspection device 100b, post-assembly inspection device 100d, and post-reflow inspection device 100f (hereinafter collectively referred to as the inspection device 30) are connected to the teaching terminal 10 and the database 20 (hereinafter also referred to as DB 20) via a network such as a LAN. The teaching terminal 10 may include a general-purpose computer system having a central processing unit (CPU) (processor), a main memory device (memory), an auxiliary memory device (hard disk, etc.), an input device (keyboard, mouse, controller, touch panel, etc.), an output device (display, printer, speaker, etc.), etc. Or it may also include a portable computer system such as a tablet terminal.

Furthermore, the inspection in the inspection device 30 of the production line 100 is performed using a pre-learned AI model (hereinafter also referred to as an AI model). The AI model is stored in the DB 20 and is used for the inspection in the inspection device 30 after appropriate selection. Moreover, the AI model is learned using the teaching terminal 10 so that the quality of the printed circuit board can be determined in advance in the inspection device 30 based on the learning data. Thereafter, the AI model is evaluated for performance based on the test data, the library data, and the mass production data.

Figure 2 (a) and Figure 2 (b) show the characteristics of each data of learning data, test data, library data, and mass production data. The horizontal axis of Figure 2 (a) and Figure 2 (b) is the period of data collection, and the vertical axis is the range of data deviation. Here, the learning data is the image data used when the AI is learning. It is the data selected from the mass production data of a very short period, and the range of period and deviation is limited to a small range. In addition, the test data is the data used for testing the pre-learning AI model. Although it also depends on the method of obtaining the data, it is generally believed that the range of deviation and period is small. The library data is further based on a large range of mass production data, for example, data collected from a large group including data of other batches of products and products produced by other bases. Moreover, mass production data is data obtained in the actual mass production process, and is data with a sufficiently large range of deviations and periods. Furthermore, there are various patterns in the relationship between each data of learning data, test data, and library data. As shown in (a) of Figure 2, there are cases where each data is independent, and as shown in (b) of Figure 2, there are cases where part or all of each data is repeated, such as when learning data overlaps with library data, or when test data overlaps with library data.

Here, since the AI model is first trained using learning data with a small range of deviations and periods of overall data, it may not be able to fully demonstrate its performance in the inspection of mass-produced data with a large range of deviations and periods of data. For any inspection, there are parts where it is not known which AI model is suitable until it is applied, and the AI model needs to be replaced and re-learned as needed. As mentioned above, it is not easy to improve the inspection accuracy in each production line at an early stage.

Therefore, in the present invention, for multiple AI models, users can view the specifications when they were generated or the specification information indicating what kind of learning data was used for learning, and the performance information indicating what kind of performance was exerted on the test data, library data, and mass production data. In this way, the best AI model can be applied to mass production more easily.

Next, an example of the processing sequence of the inspection system 1 of the present invention is described using the flowchart of Figure 3. After the start of this process, first, in step S101, the user selects a model group of parts that are the object of inspection using the AI model in the teaching terminal 10. Then, in step S102, the inspection logic corresponding to the inappropriate content that is the object of inspection using the AI model is selected. As a result, in step S103, the AI model management screen 21 (described below) is displayed on the teaching terminal 10, and the AI model management screen 21 displays information on the AI model used by the inspection logic for the model group selected in steps S101 and S102.

Then, in step S104, the user determines whether to generate a new AI model while viewing the AI model management screen 21. If the AI model is to be generated, the process proceeds to step S105 to generate a new AI model. On the other hand, if the AI model is not to be generated, the process proceeds to step S106.

Next, in step S106, the user determines whether to import the AI model from outside the system. If the user determines to import the AI model from outside the system, the process proceeds to step S107 to import the AI model from outside. If the user determines not to import the AI model from outside the system, the process proceeds to step S108.

In step S108, the user determines whether to test all AI models for the test data and the library data. Here, if the user determines to test all AI models, the process proceeds to step S109. If the user determines not to test all AI models, the process proceeds to step S112.

In step S109, all AI models listed in the AI model management screen 21 are tested for the test data and library data. After the processing of step S109 is completed, step S110 is entered. In step S110, the test results for the test data and library data are displayed on the AI model management screen 21, and the test results are saved together with the test date and time. Furthermore, the test results saved at this time are displayed the next time the AI model management screen is displayed in S103. In step S111, the user determines whether to change the AI model used for mass production inspection. If it is determined that the AI model used for mass production inspection is changed, step S112 is entered to change the AI model used for mass production inspection. On the other hand, without changing the AI model used for mass production inspection, the process proceeds to step S113 to implement mass production inspection. After the processing of step S113 is completed, this routine is temporarily terminated.

<Functional Block Diagram> FIG4 shows a functional block diagram of the inspection system 1 of the present invention. The inspection system 1 includes a teaching terminal 10, a DB 20, and an inspection device 30. The teaching terminal 10 includes an AI model generation information display unit 10a that displays information such as learning data or generation date and time when generating an AI model, and an AI model generation unit 10b that generates an AI model. In addition, the teaching terminal 10 includes an AI model management information display unit 10c that has the function of displaying historical information on the application of the AI model to mass production and performance information for test data or mass production data, and an AI model test processing unit 10d that provides various image data to the AI model and performs tests. The AI model management information display unit 10c is equivalent to the management information display unit in this embodiment.

DB 20 includes an AI model body/limit value storage unit 20a for storing the AI model body and the check limit value, an AI model management information storage unit 20b for storing the specification information of the generated AI model, i.e., the learning data, the generation date and time, and the mass production result information storage unit 20c for storing the performance information of the mass production data. In addition, DB 20 includes a learning data image storage unit 20d for storing the image data used for learning the AI model, a test data image storage unit 20e for storing the image data for testing the AI model, and a library data image storage unit 20f for storing the image data for library testing of the AI model. Furthermore, the image storage unit 20d for learning data, the image storage unit 20e for test data, and the image storage unit 20f for library data store defective types (defective names) of the image data. The missed rate is calculated as the ratio of samples of the defective type that are the object of the inspection logic that are missed, so image data of the defective type is required.

Furthermore, the inspection device 30 includes a mass production inspection section 30a that performs inspection using image data acquired during mass production.

(AI model generation stage) In FIG. 4 , arrows indicate the transfer of information of the inspection system 1 in the stage of generating an AI model. In the stage of generating an AI model, first, the learning data for learning the AI model is input to the AI model generation unit 10b from the learning data image storage unit 20d. Then, in the AI model generation unit 10b, the AI model is made to learn, thereby generating a learned AI model. Information related to the generated AI model is input from the AI model generation unit 10b to the AI model generation information display unit 10a, and the information of the generated AI model is displayed in the AI model generation information display unit 10a. Then, the AI model generation information display unit 10a inputs the generated AI model to the AI model body/threshold value storage unit 20a. Furthermore, the information such as the learning data and generation date of the generated AI model is input into the AI model management information storage unit 20b. Furthermore, here, the information input into the AI model management information storage unit 20b is equivalent to the specification information in this embodiment, and is associated with the AI model data and stored.

(AI model test phase) Next, the information transfer of the inspection system 1 in the AI model test phase is explained using FIG5. In the AI model test phase, the test data used for model testing is input from the test data image storage unit 20e to the AI model test processing unit 10d. Moreover, the library data used for model testing is input from the library data image storage unit 20f to the AI model test processing unit 10d. Furthermore, the AI model to be model tested is input from the AI model body/threshold value storage unit 20a to the AI model test processing unit 10d.

Furthermore, the results of the model test on the test data and library data are input from the AI model test processing unit 10d to the AI model management information display unit 10c, and the test results are displayed in the AI model management information display unit 10c. At this time, the information of the AI model, such as the learning data and the generation date and time of the AI model, is delivered from the AI model management information storage unit 20b to the AI model management information display unit 10c for display. Furthermore, the performance of the AI model based on the actual mass production results is input from the mass production result information storage unit 20c to the AI model management information display unit 10c for display.

(Mass production inspection stage) Next, the transfer of information of the inspection system 1 in the mass production inspection stage is explained using FIG6. In the mass production inspection stage, the selected AI model body and the inspection limit value in the mass production inspection are input to the inspection device 30 from the AI model body/limit value storage unit 20a of the database 20. Then, in the mass production inspection unit 30a, the mass production inspection is implemented using the selected AI model. Next, the mass production inspection result data is input from the mass production inspection unit 30a to the mass production result information storage unit 20c. In the mass production result information storage unit 20c, the mass production inspection result data is stored in association with the AI model used.

The display shown in FIG7 is a setting screen display 11 for setting the critical value related to the quality judgment of each inspection item in the inspection device 30. Among them, for example, regarding the wettability as an inspection item, it is displayed as an inspection using AI. And by clicking the setting button 11a of the AI inspection logic set on the right side of the critical value display window, the AI model management screen 21 described below can be displayed.

Next, FIG8 shows an example of an AI model management screen 21 displayed by the AI model management information display unit 10c. In the AI model management screen 21, an inspection logic display area 21a is configured at the upper end. Here, the management screen for the AI model for which inspection item is recorded. In the example of FIG8, the management screen for the AI model used for the wettability of the solder is shown. Moreover, at the lower left end of the inspection logic display area 21a in the AI model management screen 21, a model group display area 21b indicating which type of parts the AI model is related to the inspection is provided.

In this example, management information related to model group R1005 is shown. In addition, a display switching button area 21c is provided on the right side of the model group display area 21b. The display switching button area 21c displays a button for switching the AI model management screen of the model group unit or the overall AI model management screen including multiple model groups. In this example, the column of the model group unit is displayed in a darker color, indicating that the AI model management screen of the model group unit is selected.

In addition, an AI usage history display area 22 is provided in the AI model management screen 21. The AI usage history display area 22 is an area for displaying the usage history of each AI model in mass production. The model name of the AI model used in mass production is recorded at the left end 22b of the AI usage history display area 22. The period during which the model was used is displayed in a bar graph in the column recording each model name. A date area 22a recording the date on which the AI model was used is provided in the upper column of the AI usage history display area 22. The AI usage history display area 22 can display historical information of the period during which each model was used in mass production. In the present embodiment, this historical information is equivalent to the status applied to mass production. Furthermore, by clicking the button of the model name column at the upper left end of the AI usage history display area 22, a sorting menu is displayed, and the display method of the AI usage history display area 22 can be selected. For example, there is a sorting function such as only displaying AI models with a record of being used in mass production inspection, or displaying models with a newer generation date at the top.

Moreover, the AI model management screen 21 includes a performance display area 23 on the right side of the AI usage history display area 22. For each AI model, the performance for the test data is displayed in the left column 23a of the performance display area 23. Moreover, for each AI model, the performance for the library data is displayed in the right column 23b of the performance display area 23. More specifically, the over-view rate and the missed rate when the test data is used, and the over-view rate and the missed rate when the library data is used are displayed in a bar graph. Furthermore, the performance data in the performance display area 23 can be the result of a test executed in real time while the AI model management screen 21 is displayed, or it can be the result of a test performed in the past. The execution date and time of the model test is displayed on the upper side of the performance display area 23. The information displayed in the performance display area 23 is equivalent to performance information in this embodiment, which is equivalent to the performance of the AI model for model testing before mass production application. In addition, the over-viewing rate shown in this embodiment is equivalent to the over-viewing index, and the missed viewing rate is equivalent to the missed viewing index.

Furthermore, a mass production performance display area 24 is provided below the AI usage history display area 22. The mass production performance display area 24 displays the performance of the AI model used on each working day shown in the AI usage history display area 22 for mass production data. More specifically, the over-view rate and the missed-view rate are displayed in a bar graph. The information displayed in the mass production performance display area 24 is equivalent to performance information in this embodiment, which is equivalent to the performance when applied to mass production.

Moreover, a detailed display button 21d is provided at the right end of the AI model management screen 21 corresponding to the model name of each AI model. By pressing the detailed display button 21d, detailed information of each AI model is displayed. FIG9 shows an example of a first detailed display screen 31a displayed by pressing the detailed display button 21d. A display mode selection area 32 is displayed on the first detailed display screen 31a, and in the display mode selection area 32, the display content in the detailed display screen can be selected. In FIG9, basic information is selected in the display mode selection area 32. Thereby, the first detailed display screen 31a becomes a screen for displaying basic information.

As shown in FIG9 , a basic information display area 33 is provided in the first detailed display screen 31a. In the basic information display area 33, the model name of the target AI model, the date and time of generation, the image size of the image to be inspected, the network name used, the name of the learning data, the recycling period of the learning data, the batch number of the product to be inspected, and the comments are recorded. In this embodiment, the information shown in the basic information display area 33 is equivalent to the specification information. In addition, a detail button 33a is provided in the field of the learning data name in the basic information display area 33 of FIG9 , and the detail button 33a is used to retrieve more detailed information related to the amount of learning data used in learning. By clicking the detail button 33a, information on the amount of learning materials used in learning for each model as shown in FIG. 10(a) and FIG. 10(b) can be viewed.

FIG10(a) is a learning data number display screen 330 that displays the number of learning data used in learning for each model when learning in the learning model in the detailed display is unsupervised learning. The learning data number display screen 330 is provided with a learning data number area 330a that displays the number of learning data used in learning in a list for each model.

Here, since the learning model can be trained only with good product data, the number of learning data displayed is the number of good product data. In this way, by listing the number of learning data used for each model, it is possible to quickly check whether the number of learning data for each model is biased or whether the number of learning data for each model is sufficient.

In addition, in the learning data number area 330a, it is also possible to emphasize the model with extremely small learning data by changing the display color. For example, in the example of (a) in FIG. 10, since the learning data for model D is extremely small, it can be understood that the learning model may not be able to fully demonstrate the performance of model D. As a method of this emphasis display, in addition to the method of changing the display color, a method of changing the thickness or size of the text, a method of changing the color of the column of the list, etc. can also be adopted. Moreover, as a condition for emphasizing the display, for example, a case where the number of data is less than a specified ratio (for example, 10%) relative to the maximum value of each model in the learning data number area 330a can be illustrated.

Next, FIG. 10 (b) shows a learning data number display screen 331, which displays the number of learning data used in learning for each model when the learning of the learning model in the detailed display is supervised learning. In this learning data number display screen 331, a learning data number area 331a is provided, and the learning data number area 331a displays the number of learning data and the number of verification data used in learning for each model. In this learning data number area 331a, the number of learning data and the number of verification data are divided into the number of good product data and the number of defective product data, respectively.

With this, when supervised learning of a learning model in detailed display, it is possible to check how much learning data and verification data are used by dividing the number of good product data and the number of defective product data for each model, and to check the traceability of learning data in more detail. In this case, it is not necessary to display the number of verification data. In addition, the number of verification data can also display the number of test data or library data in addition to the data during learning.

Next, the second detailed display screen 31b displayed when "Model test results for test data" is selected in the display mode selection area 32 is described using Fig. 11. In the second detailed display screen 31b, a table showing the model test results for test data, i.e., a test result table 34, is arranged at the lower center of the display mode selection area 32.

The left end of the test result table 34 shows the type of image data used in the model test (i.e., image data of good products, image data of defective products, or image data of gray products). In addition, the upper section records the number of image data of each type judged as good products, the number judged as defective products, the over-view rate, or the missed-view rate by the AI model. In addition, the image display area 35 showing how each actual image data is judged by the AI model is arranged on the left side of the lower section of the second detailed display screen 31b.

In the image display area 35, the actual image data is displayed on the left end, and the image data obtained by heat mapping the image data, the visual quality difference, the judgment result obtained by using the AI model, and the AI output value at that time are displayed in each row in the column of the image data. In addition, a histogram area 36 is provided on the right side of the lower section of the second detailed display screen 31b, and the histogram area 36 displays a histogram of the AI output value when testing each image data with the horizontal axis as the critical value of quality. Through the second detailed display screen 31b, the test results of the AI model of the object can be confirmed in detail.

Here, the structure of the third detailed display screen displayed when "model test results for library data" is selected in the display mode selection area 32 is basically the same as the second detailed display screen 31b, so the description is omitted.

FIG12 shows a fourth detailed display screen 31c displayed when "mass production results" is selected in the display mode selection area 32. The basic structure of the fourth detailed display screen 31c is the same as that of the second detailed display screen 31b, but in this example, a mass production information area 37 for displaying the model usage period and the ID of the inspected substrate is provided to cope with mass production, which is different from the second detailed display screen 31b.

Next, we return to the description of FIG. 8 for the moment, but a model generation button 21e, a model test button 21f, an import button 21g, and a model application button 21h are provided in the lower right area of the AI model management screen 21. By clicking the model generation button 21e, generation of a new AI model can be started. By clicking the model test button 21f, a test using a specific AI model can be performed immediately using, for example, test data or library data. Moreover, by clicking the import button 21g, a new AI model can be imported from the outside. Furthermore, by clicking the model application button 21h, an AI model that is not currently applied to mass production can be applied to mass production.

Furthermore, in the display switching button area 21c, in addition to the display of the model group unit described so far, it is possible to select to display the entire model. FIG. 13 shows a display screen when the entire model is selected in the display switching button area 21c, that is, an AI model overall management screen 41. The AI model overall management screen 41 is provided with a model group status display area 45 that indicates the status of applying the AI model to a plurality of model groups in the production line 100. The names of the model groups are listed at the left end 45a of the model group status display area 45.

In addition, for each model group part, whether the inspection using the AI model is enabled, the threshold value, the AI model name, and the performance (over-read rate or under-read rate) of mass production using the AI model are recorded. Furthermore, even if the inspection using AI is enabled, if the AI model is not defined, it will cause an inspection error, so the column can change its own color to notify the user. In FIG13, the columns that meet the conditions in the model group status display area 45 are hatched.

Furthermore, a model group expansion button 45b is provided on the right side of each model group row in the model group status display area 45. The model group expansion button 45b can expand the AI model of the model with good performance to other model groups by simple operation. After clicking the button, the model group of the expansion target is set, thereby expanding the AI model to other model groups.

Furthermore, in the above-described embodiment, the present invention has been described as being applied to the production line 100 of printed circuit boards, but of course, the present invention can be applied to other types of production lines. Moreover, the present invention can also be regarded as a data set including AI model data, specification information, and performance information that can display the AI model management screen 21 and the first detailed display screen 31a to the fourth detailed display screen 31c. Furthermore, the data set can also include an inspection image that serves as the basis for performance information, or an inspection image that can be inspected in real time using AI model data.

Furthermore, in order to compare the structural elements of the present invention with the structure of the embodiment, the symbols of the structural element annotation diagram of the present invention are annotated below. <Annotation 1> An inspection system (1) uses an AI model to perform inspection based on image data of the product in a production line (100) of the product. The inspection system (1) is characterized by comprising: A management information display unit (10c) that displays information related to the management of one or more AI models used for the inspection of the product. The management information display unit (10c) can display specification information related to the specifications of the AI model and performance information related to the performance of the AI model for various data. The performance information includes the performance of the AI model for model testing before mass production application and the performance when applied to mass production. <Note 11> A method for managing AI model data, for use in an inspection system (1) that uses an AI model in a production line (100) of a product to perform inspection based on image data of the product, The method for managing AI model data is characterized by: The AI model data is managed by associating specification information related to the specifications of the AI model data and performance information related to the performance of the AI model for various data, The performance information includes the performance of the AI model for model testing before mass production application and the performance when applied to mass production. <Note 16> An AI model data set, comprising one or more AI model data, is used in an inspection system (1) that uses an AI model in a production line (100) of a product to perform inspection based on image data of the product. The AI model data set is characterized in that: It includes specification information related to the specifications of the AI model data and performance information related to the performance of the AI model for various data, and The performance information includes each inspection image and the judgment result of the AI model on the quality of the inspection image.

1: Inspection system 10: Teaching terminal 10a: AI model generation information display unit 10b: AI model generation unit 10c: AI model management information display unit 10d: AI model test processing unit 11: Setting screen display 11a: AI inspection logic setting button 20: DB 20a: AI model body/limit value storage unit 20b: AI model management information storage unit 20c: Mass production result information storage unit 20d: Learning data image storage unit 20e: Test data image storage unit 20f: Library data image storage unit 21: AI model management screen 21a: Inspection logic display area 21b: Model group display area 21c: Display switching button area 21d: Detailed display button 21e: Model generation button 21f: Model test button 21g: Import button 21h: Model application button 22: AI usage history display area 22a: Date area 22b: Left end of AI usage history display area 23: Performance display area 23a: Left column of performance display area 23b: Right column of performance display area 24: Mass production performance display area 30: Inspection device 30a: Mass production inspection department 31a: First detailed display screen 31b: Second detailed display screen 31c: Fourth detailed display screen 32: Display mode selection area 33: Basic information display area 33a: Detailed button 34: Test result table 35: Image display area 36: Histogram area 37: Mass production information area 41: AI model overall management screen 45: Model group status display area 45a: Left end of model group status display area 45b: Model group expansion button 100: Production line 100a: Solder printing device 100b: Post-solder printing inspection device 100c: Assembly machine 100d: Post-assembly inspection device 100e: Reflow furnace 100f: Post-reflow inspection device 330,331: Learning data number display screen 330a,331a: Learning data number area

FIG. 1 is a schematic diagram of the inspection system of the embodiment of the present invention. FIG. 2 is a diagram showing the characteristics of each data of learning data, test data, library data, and mass production data. FIG. 3 is a flowchart of the processing using the inspection system of the embodiment of the present invention. FIG. 4 is a functional block diagram of the inspection system of the embodiment of the present invention. FIG. 5 is a second diagram of the functional block diagram of the inspection system of the embodiment of the present invention. FIG. 6 is a third diagram of the functional block diagram of the inspection system of the embodiment of the present invention. FIG. 7 is a diagram showing a screen display for setting of the embodiment of the present invention. FIG. 8 is a diagram showing an example of an AI model management screen of the embodiment of the present invention. FIG. 9 is a diagram showing the first detailed display screen of the embodiment of the present invention. FIG. 10 is a diagram showing a learning data number display screen of an embodiment of the present invention. FIG. 11 is a diagram showing a second detailed display screen of an embodiment of the present invention. FIG. 12 is a diagram showing a fourth detailed display screen of an embodiment of the present invention. FIG. 13 is a diagram showing an AI model overall management screen of an embodiment of the present invention.

21: AI model management screen

21a: Check the logical display area

21b: Model group display area

21c: Display switch button area

21d:Detailed display button

21e: Model generation button

21f: Model test button

21g: Import button

21h: Model application button

22: AI usage history display area

22a: Date area

22b: Left end of AI usage history display area

23: Performance display area

23a: Left column of the performance display area

23b: Right column of the performance display area

24: Mass production performance display area

Claims (14)

An inspection system, in a production line of a product, uses an artificial intelligence model to perform inspection based on image data of the product, wherein the inspection system includes a management information display unit, the management information display unit displays information related to the management of one or more artificial intelligence models used for the inspection of the product, the management information display unit can display specification information related to the specifications of the artificial intelligence model, and performance information related to the performance of the artificial intelligence model for various data, the performance information includes the performance of the artificial intelligence model for model testing before mass production application, and the performance when applied to mass production, wherein the management information display unit can display the performance information for each inspection item for the product. An inspection system as described in claim 1, wherein the management information display unit displays the performance of the artificial intelligence model for model testing before mass production application, the performance when applied to mass production, and the status of application to mass production as the performance information on one screen. The inspection system as described in claim 1, wherein the model testing of the artificial intelligence model before mass production application includes at least one of model testing on test data or model testing on library data. An inspection system as described in claim 1, wherein the performance includes an over-reading indicator of the number of good products judged as defective products in an inspection without using the artificial intelligence model based on the artificial intelligence model, and an under-reading indicator of the number of defective products judged as good products in an inspection without using the artificial intelligence model based on the artificial intelligence model. The inspection system as described in claim 1, wherein the performance information further includes the model test of the artificial intelligence model before mass production application and each inspection image in mass production, the judgment result of the artificial intelligence model on the quality of the inspection image, and the artificial intelligence output value related to the inspection image. An inspection system as described in claim 5, wherein the performance information further includes a histogram of the artificial intelligence output values of the artificial intelligence model with respect to the inspection image. An inspection system as described in any one of claims 1 to 6, wherein the management information display unit is capable of switching the display of the specification information and the performance information by displaying each model group of the inspection target parts and displaying a summary of multiple model groups. An inspection system as described in claim 7, wherein the management information display unit can perform processing of applying an artificial intelligence model applied to one model group to other model groups in a display screen that summarizes multiple model groups. An inspection system as described in any one of claims 1 to 6, wherein the specification information includes at least one of network information of the artificial intelligence model, information of learning data during learning, and a batch number of the artificial intelligence model. A method for managing artificial intelligence model data is used in an inspection system that uses artificial intelligence models to perform inspections based on image data of products in a production line of products. The method for managing artificial intelligence model data is characterized in that the artificial intelligence model data is managed by establishing associations with specification information related to the specifications of the artificial intelligence model data and performance information related to the performance of the artificial intelligence model for various data, wherein the performance information includes the performance of the artificial intelligence model for model testing before mass production and the performance when applied to mass production. The method for managing artificial intelligence model data displays the specification information related to the specifications of the artificial intelligence model data and the performance information related to the performance of the artificial intelligence model for various data, and displays the performance information for each inspection item for the product. A method for managing artificial intelligence model data as described in claim 10, wherein the model test before mass production application of the artificial intelligence model includes at least one of a model test on test data or a model test on library data. A method for managing artificial intelligence model data as described in claim 10, wherein the performance includes an over-reading indicator of the number of good products judged as defective products in an inspection without using the artificial intelligence model based on the artificial intelligence model, and an under-reading indicator of the number of defective products judged as good products in an inspection without using the artificial intelligence model based on the artificial intelligence model. A method for managing artificial intelligence model data as described in claim 10, wherein the performance information further includes model testing before mass production of the artificial intelligence model and each inspection image during mass production, the artificial intelligence model's judgment result on the quality of the inspection image, and the artificial intelligence output value related to the inspection image. A method for managing artificial intelligence model data as described in any one of claim 10 to 13, wherein the specification information includes at least any one of network information of the artificial intelligence model, information of learning data during learning, and a batch number of the artificial intelligence model.

Images