CN111899239A - Image processing method and device - Google Patents

Abstract

Embodiments of the present application provide an image processing method and apparatus, which relate to the technical field of computers. The image processing method in the embodiment of the present application includes: based on the color channel parameters of each image in the image data set, clustering the images in the image data set to obtain multiple image cluster sets; selecting images from the multiple image cluster sets The largest number of target image clustering sets, based on the color channel parameters of each image in the target image clustering set, calculate the average color channel parameters of the images in the target image clustering set; The color channel parameters of the image are calibrated to obtain a calibrated image to be detected. The technical solutions of the embodiments of the present application enable the image defect localization model to achieve accurate defect localization for liquid crystal panel images gathered in a specific color mode, thereby improving the accuracy of defect localization by the image defect localization model.

Description

Image processing method and device

technical field

The present application relates to the field of computer technology, and in particular, to an image processing method and apparatus.

Background technique

In the manufacturing process of the liquid crystal panel, it is necessary to photograph the liquid crystal panel processed by each processing process to obtain the image of the liquid crystal panel, and process the image of the liquid crystal panel through the image defect positioning model to obtain the defect position of the liquid crystal panel image. .

The image defect localization model needs to be trained by a large number of LCD panel images. In the related art, the LCD panel image captured during the manufacturing process of the LCD panel is generally used as the training sample data of the image defect localization model to train the image defect localization model.

Since the shooting of the LCD panel will be affected by various factors such as ambient light, camera imaging parameters, and the reflectivity of the LCD panel, the LCD panel images trained on the image defect localization model show a specific color pattern aggregation, such as light yellow, Yellow-green, brown-yellow or orange-yellow, etc. Using these LCD panel images directly as the training sample data for the image defect localization model will result in the image defect localization model being unable to achieve accurate defect localization for the LCD panel images gathered in a specific color mode, and the defect localization accuracy is not high.

SUMMARY OF THE INVENTION

The embodiments of the present application provide an image processing method and device, which can to a certain extent solve the problem that the image defect localization model cannot achieve accurate defect localization for liquid crystal panel images gathered in a specific color mode, and the accuracy of defect localization is not high. question.

Other features and advantages of the present application will become apparent from the following detailed description, or be learned in part by practice of the present application.

According to an aspect of the embodiments of the present application, an image processing method is provided, comprising: performing clustering processing on images in the image data set based on color channel parameters of each image in the image data set to obtain a plurality of image clustering sets Select the target image cluster set with the largest number of images from the multiple image cluster sets, calculate the average color channel parameters of the images in the target image cluster set based on the color channel parameters of each image in the target image cluster set ; Based on the average color channel parameters of the images in the target image cluster set, the color channel parameters of the images to be detected are calibrated to obtain the calibrated images to be detected.

According to an aspect of the embodiments of the present application, an image processing apparatus is provided, including: a clustering unit configured to perform clustering processing on the images in the image data set based on the color channel parameters of each image in the image data set, to obtain a plurality of image clustering sets; a computing unit for selecting a target image clustering set with the largest number of images from the multiple image clustering sets, and calculating the The average color channel parameter of the images in the target image clustering set; the first calibration unit is configured to perform calibration processing on the color channel parameter of the image to be detected based on the average color channel parameter of the image in the target image clustering set, and obtain the calibrated to-be-detected image. Detect images.

In some embodiments of the present application, based on the foregoing solution, the clustering unit is configured to: based on the color channel parameters corresponding to all pixels included in each image in the image data set, separately calculate each pixel in the image data set. The average color channel parameters corresponding to each image; and based on the average color channel parameters corresponding to each image in the image data set, the images in the image data set are clustered to obtain a plurality of image cluster sets.

In some embodiments of the present application, based on the foregoing solution, the clustering unit is configured to: based on the color channel parameters of all pixels included in each image in the image dataset under each color channel, calculate the The average color channel parameters under each color channel for each image in the image dataset.

In some embodiments of the present application, based on the foregoing solution, the first calibration unit is configured to: determine, based on color channel parameters of all pixels included in the image to be detected, an average color channel parameter corresponding to the image to be detected; The average color channel parameters of the images in the target image cluster set and the average color channel parameters corresponding to the images to be detected generate calibration parameters; based on the calibration parameters, the colors of all pixels included in the images to be detected are analyzed The channel parameters are calibrated to obtain a calibrated image to be detected.

In some embodiments of the present application, based on the foregoing solution, the first calibration unit is configured to: calculate the average color channel parameter of the images in the target image cluster set and the average color channel parameter corresponding to the image to be detected A ratio between; based on the ratio, a calibration parameter is generated.

In some embodiments of the present application, based on the foregoing solution, the image processing apparatus further includes: an input unit for inputting the calibrated to-be-detected image into a pre-trained image defect localization model; a localization unit for using Defect localization processing is performed on the calibrated image to be inspected through the image defect localization model, and a defect localization result is output.

In some embodiments of the present application, based on the foregoing solution, the image processing apparatus further includes: a second calibration unit, configured to perform a calibration on the image data set based on the average color channel parameters of the images in the target image cluster set The color channel parameter of the image is calibrated to obtain a calibrated image data set; the labeling unit is used to perform image defect labeling processing on the images in the calibrated image data set to obtain the labeled image data set; The generating unit , which is used to generate training sample data based on the marked image data set; the training unit is used to train the machine learning model based on the generated training sample data to obtain the image defect localization model.

In some embodiments of the present application, based on the aforementioned solution, the generating unit is configured to: select a target image from the labeled image data set; perform data enhancement processing on the target image to obtain a processed image ; Generate training sample data based on the processed image and the labeled image data set.

In some embodiments of the present application, based on the foregoing solution, the generating unit is configured to: obtain a data enhancement probability threshold; assign random numbers to the images in the labeled image dataset; assign the assigned random numbers less than or equal to The image with the data enhancement probability threshold is determined as the target image.

According to an aspect of the embodiments of the present application, a computer-readable medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, implements the image processing method described in the foregoing embodiments.

According to an aspect of the embodiments of the present application, an electronic device is provided, including: one or more processors; and a storage device for storing one or more programs, when the one or more programs are stored by the one or more programs When executed by multiple processors, the one or more processors are made to implement the image processing method described in the above embodiments.

In the technical solutions provided by some embodiments of the present application, the images in the image data set are clustered based on the color channel parameters of each image in the image data set, so as to obtain multiple image clustering sets, and from multiple image clustering sets, The target image cluster set with the largest number of images is selected in the cluster set, and based on the color channel parameters of each image in the target image cluster set, the average color channel parameters of the images in the target image cluster set are calculated, and then the average color channel parameters of the images in the target image cluster set are calculated. Average color channel parameters, calibrate the color channel parameters of the image to be inspected, and obtain the calibrated image to be inspected. First, perform image calibration data preprocessing on the image to be inspected, and then use the image defect localization model according to the calibrated image to be inspected. Compared with directly inputting the image to be inspected without data preprocessing without image calibration into the image defect localization model, when the image to be inspected is an LCD panel image gathered in a specific color mode, the image defect can be located The model can also perform accurate defect location, which improves the accuracy of defect location by the image defect location model.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not limiting of the present application.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description serve to explain the principles of the application. Obviously, the drawings in the following description are only some embodiments of the present application, and for those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative effort. In the attached image:

FIG. 1 shows a schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of the present application can be applied.

FIG. 2 shows a flowchart of an image processing method according to an embodiment of the present application.

FIG. 3 shows a specific flowchart of step S210 of the image processing method according to an embodiment of the present application.

FIG. 4 shows a specific flowchart of step S230 of the image processing method according to an embodiment of the present application.

FIG. 5 shows a specific flowchart of step S410 of the image processing method according to an embodiment of the present application.

FIG. 6 shows a flowchart of an image processing method according to an embodiment of the present application.

FIG. 7 shows a flowchart of an image processing method according to an embodiment of the present application.

FIG. 8 shows a specific flowchart of step S720 of the image processing method according to an embodiment of the present application.

FIG. 9 shows a specific flowchart of step S810 of the image processing method according to an embodiment of the present application.

FIG. 10 shows a block diagram of an image processing apparatus according to an embodiment of the present application.

FIG. 11 shows a schematic structural diagram of a computer system suitable for implementing the electronic device according to the embodiment of the present application.

Detailed ways

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments, however, can be embodied in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art.

Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided in order to give a thorough understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical solutions of the present application may be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be employed. In other instances, well-known methods, devices, implementations, or operations have not been shown or described in detail to avoid obscuring aspects of the present application.

The block diagrams shown in the figures are merely functional entities and do not necessarily necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices entity.

The flowcharts shown in the figures are only exemplary illustrations and do not necessarily include all contents and operations/steps, nor do they have to be performed in the order described. For example, some operations/steps can be decomposed, and some operations/steps can be combined or partially combined, so the actual execution order may be changed according to the actual situation.

FIG. 1 shows a schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of the present application can be applied.

As shown in FIG. 1 , the system architecture may include a client 101 , a network 102 and a server 103 . The medium used by the network 102 to provide the communication link between the client 101 and the server 103 . The network 102 may include various connection types, such as wired communication links, wireless communication links, and the like.

It should be understood that the numbers of clients 101 , networks 102 and servers 103 in FIG. 1 are merely illustrative. According to implementation requirements, there may be any number of clients 101 , networks 102 and servers 103 , for example, the server 103 may be a server cluster composed of multiple servers, and the like.

The client 101 interacts with the server 103 through the network 102 to receive or send messages, etc. The server 103 may be a server that provides various services, such as a server that provides image processing applications.

The client 101 performs clustering processing on the images in the image dataset based on the color channel parameters of each image in the image dataset to obtain multiple image cluster sets, and selects the target image cluster with the largest number of images from the multiple image cluster sets. class set, and based on the color channel parameters of each image in the target image cluster set, calculate the average color channel parameters of the images in the target image cluster set, and then based on the average color channel parameters of the images in the target image cluster set, the color channel of the image to be detected is calculated. The parameters are calibrated to obtain the calibrated image to be inspected. First, the image to be inspected is preprocessed for image calibration, and then the image defect location model is used to perform defect location detection according to the calibrated image to be inspected. The image to be inspected that has undergone image calibration data preprocessing is directly input to the image defect localization model, so that when the image to be inspected is an LCD panel image gathered in a specific color mode, the image defect localization model can also perform accurate defect localization. The accuracy of defect localization by the image defect localization model.

It should be noted that the image processing method provided by the embodiment of the present application is generally executed by the client 101 , and accordingly, the image processing apparatus is generally set in the client 101 . However, in other embodiments of the present application, the server 103 may also have similar functions to the client 101, so as to execute the solution of the image processing method provided by the embodiments of the present application.

The implementation details of the technical solutions of the embodiments of the present application are described in detail below.

FIG. 2 shows a flowchart of an image processing method according to an embodiment of the present application. The image processing method may be executed by a client, and the client may be the client 101 shown in FIG. 1 . Referring to FIG. 2 , the image processing method includes at least steps S210 to S230, which are described in detail as follows.

In step S210, based on the color channel parameters of each image in the image data set, cluster processing is performed on the images in the image data set to obtain a plurality of image cluster sets.

In one embodiment, in the manufacturing process of the liquid crystal panel, after the liquid crystal panel is processed by a certain processing process, a specific image capturing device can be used to capture the processed liquid crystal panel to obtain an image of the liquid crystal panel, and the obtained liquid crystal panel can be captured by capturing the image of the liquid crystal panel. The panel image is used as an image for color defect analysis of the liquid crystal panel. The image data set is a collection of images obtained by shooting the LCD panel. It should be pointed out that due to the objective differences in the shooting environment where each LCD panel image is located, for example, due to various factors such as ambient lighting, camera imaging parameters, and the reflectivity of the LCD panel. Differences in the shooting environment caused by different factors lead to differences in the color of the LCD panel image. Therefore, the liquid crystal panel image set included in the image data set generally refers to the liquid crystal panel image set with differences in color characteristics.

In one embodiment, the color channel parameter of the image is used as the parameter value in the color channel corresponding to the image in a specific color mode, and the color channel parameter of the image can be used as a parameter that characterizes the color characteristics of the image. The image generally contains multiple Each pixel has corresponding color channel parameters, and the color channel parameters of all pixels included in the image constitute the color channel parameters of the image.

The color mode corresponding to the image may be the color mode of a single color channel, such as a grayscale image. For an image with a single color channel, the color channel parameter value corresponding to each pixel has only the parameter value under one color channel. The color mode corresponding to the image can also be a color mode with multiple color channels, such as the red, green, and blue RGB color mode. For example, in the RGB color mode, the corresponding color channel parameters are three parameter values, that is, in the R color channel, G The color channel and the parameter value under the B color channel.

In one embodiment, in order to determine the common color characteristics of all images included in the image dataset, the images in the image dataset may be clustered based on the color channel parameters of each image in the image dataset to obtain multiple images Clustering set, each image clustering set can be used as a set of images representing the same color characteristics.

Optionally, the clustering algorithm used for clustering the images in the image dataset can be the Mean-Shift clustering algorithm or the k-means clustering algorithm, etc. Of course, it can also be other clustering algorithms, here Not limited.

Optionally, when the color channel parameters of each image in the image dataset are used as the input data for clustering processing, the color channel parameters corresponding to all pixels included in the image can be directly used as the input data for clustering to analyze the image data. The images in the data set are clustered to obtain a plurality of image cluster sets, so as to realize the clustering of the images in the image data set according to the color characteristics contained in the images.

Referring to FIG. 3, FIG. 3 shows a specific flowchart of step S210 of the image processing method according to an embodiment of the present application. In this embodiment, step S210 may specifically include steps S310 to S320, which are described in detail as follows.

In step S310, based on the color channel parameters corresponding to all pixels included in each image in the image data set, the average color channel parameters corresponding to each image in the image data set are calculated respectively.

In one embodiment, when performing clustering processing on the images in the image dataset based on the color channel parameters of each image in the image dataset, the color channel parameters corresponding to all the pixels included in each image in the image dataset may also be firstly processed. , respectively calculate the average color channel parameters corresponding to each image in the image data set, in other words, for each image in the image data set, according to the color channel parameters corresponding to all the pixels contained in the image, calculate the The corresponding average color channel parameter, the average color channel parameter is used as a parameter reflecting the overall color characteristics of the image.

Specifically, the color channel parameters of all pixels can be added up and divided by the number of pixels included in the image to obtain the average color channel parameter corresponding to the image.

Optionally, when the color mode corresponding to the image is the color mode of a single color channel, the parameters of the single color channel corresponding to all pixels contained in each image can be added and divided by the number of pixels contained in the image. number to obtain the average color channel parameter corresponding to the image, and the average color channel parameter corresponding to the obtained image is only one.

Optionally, when the color mode of the image is a color mode with multiple color channels, step S310 may specifically include: based on the color channel parameters of all the pixels included in each image in the image data set under each color channel, calculate respectively. The average color channel parameters under each color channel for each image in the image dataset.

When the color mode of the image is the color mode of multiple color channels, the average color channel parameter corresponding to the image is the color channel parameter of the multiple color channels. Taking the image in RGB color mode as an example, you can add the color channel parameters of all pixels contained in each image under the R color channel, and divide by the number of pixels contained in the image to get the image in the R color channel. In the same way, the average color channel parameters of the image in the G color channel and the average color channel parameters in the B color channel can be calculated respectively, and then the image in the R color channel, The average color channel parameters under the G color channel and the B color channel, the average color channel parameters of the image under three different color channels constitute the average color channel parameter corresponding to the image.

In step S320, based on the average color channel parameter corresponding to each image in the image data set, the images in the image data set are clustered to obtain a plurality of image cluster sets.

In one embodiment, after acquiring the average color channel parameter corresponding to each image in the image data set, it is used as the input data of the clustering algorithm, the input data is processed by the clustering algorithm, and a plurality of image clustering sets are output The clustering result of the image data set is realized, so that the images in the image data set can be clustered according to the color channel parameters of each image in the image data set.

In the technical solution of the embodiment shown in FIG. 3 , the average color channel parameters corresponding to each image in the image data set are calculated separately according to the color channel parameters corresponding to all the pixels contained in each image in the image data set, and then based on the image data set The average color channel parameters corresponding to each image in the data set, and the images in the image data set are clustered, compared to directly clustering the images in the image data set according to the color channel parameters of all pixels contained in the image. The processing method can significantly reduce the dimension of the calculation data corresponding to each image input into the clustering algorithm, thereby effectively reducing the complexity of the clustering operation, reducing the computational load of the system, and saving system resources.

Please also continue to refer to Fig. 2, in step S220, select the target image cluster set with the largest number of images from a plurality of image cluster sets, and calculate the image in the target image cluster set based on the color channel parameters of each image in the target image cluster set. The average color channel parameter of .

In one embodiment, after obtaining multiple image clustering sets, the image clustering set with the largest number of images may be selected from the multiple image clustering sets as the target image clustering set, and the target image clustering set is regarded as the most representative image clustering set. An image collection of common color properties shared by all images in an image dataset.

It can be understood that, if there are two image cluster sets with the same number of images and more images than other image cluster sets, one image cluster set can be selected from the two image cluster sets as the target image. cluster set.

In one embodiment, after the target image cluster set is determined, the average color channel parameter of the images in the target image cluster set may be calculated based on the color channel parameters of each image in the target image cluster set.

Specifically, the average color channel parameters corresponding to each image in the image data set can be calculated according to the color channel parameters corresponding to all pixels included in each image in the target image clustering set. In other words, for the target image clustering For each image in the set, calculate the average color channel parameter corresponding to the image according to the color channel parameters corresponding to all the pixels contained in the image. After calculating the average color channel parameters corresponding to each image in the target image cluster set, sum the average color channel parameters corresponding to each image in the target image cluster set, and then divide the summation result by the target The number of images in the image cluster set, and the average color channel parameters of the images in the target image cluster set are calculated.

As mentioned above, the color channel parameters of the image under each color channel need to be calculated separately, so when the color mode corresponding to the image is the color mode of a single color channel, the average color channel parameter corresponding to the obtained image is only is one, correspondingly, the average color channel parameter of the image in the target image cluster set is also only one; and when the color mode of the image is a multi-color channel color mode, the average color channel parameter corresponding to the obtained image Correspondingly, the average color channel parameters of the images in the target image cluster set are also multiple.

In step S230, a calibration process is performed on the color channel parameters of the image to be detected based on the average color channel parameters of the images in the target image cluster set to obtain a calibrated image to be detected.

In one embodiment, after determining the average color channel parameters of the images in the target image cluster set, calibration processing may be performed on the color channel parameters of the images to be detected based on the average color channel parameters of the images in the target image cluster set to obtain a calibrated image to be detected.

Specifically, the color channel parameters of all pixels included in the image to be detected can be calibrated according to the average color channel parameters of the images in the target image cluster set, so that the image to be detected can be adjusted to have the common color characteristics of all images in the image dataset , as the calibrated image to be detected.

Compared with the method of directly inputting the image to be inspected without image calibration into the image defect localization model, the data preprocessing of the image to be inspected is firstly performed on the image to be inspected, and then the image defect localization model performs the calibration according to the image to be inspected. The defect location detection enables the image defect location model to perform accurate defect location even when the image to be inspected is an LCD panel image gathered in a specific color mode, which improves the accuracy of the image defect location model for defect location.

Referring to FIG. 4 , FIG. 4 shows a specific flowchart of step S230 of an image processing method according to an embodiment of the present application. In this embodiment, step S230 may specifically include steps S410 to S430 , which are described in detail as follows.

In step S410, an average color channel parameter corresponding to the image to be detected is determined based on the color channel parameters of all pixels included in the image to be detected.

In one embodiment, when calibrating the color channel parameters of the image to be detected based on the average color channel parameters of the images in the target image cluster set, the color channel parameters of all the pixels included in the image to be detected may be first determined to determine the color channel parameters to be detected. The average color channel parameter corresponding to the detection image.

Specifically, the color channel parameters of all pixels can be added up and divided by the number of pixels included in the image to obtain the average color channel parameter corresponding to the image.

It can be understood that when the color mode corresponding to the image is the color mode of a single color channel, the parameters of the single color channel corresponding to all the pixels contained in the image to be detected can be added and divided by the pixels contained in the image. The average color channel parameter corresponding to the image is obtained by the number, and the average color channel parameter corresponding to the obtained image to be detected is only one.

When the color mode of the image to be detected is the color mode of multiple color channels, the average color channel parameter corresponding to the image to be detected is the color channel parameter of the multiple color channels. Taking an image in RGB color mode as an example, the color channel parameters of all pixels contained in the image to be detected under the R color channel can be added, and divided by the number of pixels contained in the image to be detected, to obtain the image to be detected in In the same way, the average color channel parameters of the R color channel can be calculated to obtain the average color channel parameters of the image to be detected in the G color channel and the average color channel parameters of the B color channel. The average color channel parameters of the R color channel, the G color channel, and the B color channel, and the average color channel parameters of the image to be detected under the three different color channels constitute the average color channel parameter corresponding to the image to be detected.

In step S420, calibration parameters are generated based on the average color channel parameters of the images in the target image cluster set and the average color channel parameters corresponding to the images to be detected.

In one embodiment, when generating the calibration parameters based on the average color channel parameters of the images in the target image cluster set and the average color channel parameters corresponding to the images to be detected, specifically, the average color channel parameters of the images in the target image cluster set may be based on The parameter corresponds to the average color channel parameter of the image to be detected, and the calibration parameter for calibrating the image, the average color channel parameter of the image in the target image cluster set, and the average color channel parameter corresponding to the image. Correspondence between the two relationship, generate calibration parameters for calibrating the image, and then realize the calibrating process for the image to be detected.

Referring to FIG. 5 , FIG. 5 shows a specific flowchart of step S410 of an image processing method according to an embodiment of the present application. In this embodiment, step S410 may specifically include steps S510 to S520 , which are described in detail as follows.

In step S510, the ratio between the average color channel parameter of the images in the target image cluster set and the average color channel parameter corresponding to the image to be detected is calculated.

In one embodiment, when generating the calibration parameters for calibrating the images to be detected, the ratio between the average color channel parameters of the images in the target image cluster set and the average color channel parameters corresponding to the images to be detected may be calculated first.

It can be understood that, when the color mode of the image is a multi-color channel color mode, the average color channel parameters of the images in the target image cluster set and the average color channel parameters corresponding to the images to be detected both include different color channels. There are multiple average color channel parameters, so it is necessary to calculate the ratio between the average color channel parameters of the image under different color channels.

G颜色通道对应的

以及B颜色通道对应的

Taking the image in RGB color mode as an example, the average color channel parameters of the image in the target image cluster set include the average color channel parameter R1 under the R color channel, the average color channel parameter G1 under the G color channel, and the average color channel parameter G1 under the B color channel. The color channel parameter B1, the average color channel parameter corresponding to the image to be detected includes the average color channel parameter R2 under the R color channel, the average color channel parameter G2 under the G color channel, and the average color channel parameter B2 under the B color channel, then The ratio between the average color channel parameter of the image in the target image cluster set and the average color channel parameter corresponding to the image to be detected includes the corresponding value of the R color channel.

Corresponding to the G color channel

and the corresponding B color channel

In step S520, based on the ratio, calibration parameters are generated.

In one embodiment, after obtaining the ratio between the average color channel parameter of the image in the target image cluster set and the average color channel parameter corresponding to the image to be detected, based on the ratio, the correspondence between the ratio and the calibration parameter, Generate calibration parameters for calibrating the image to be inspected.

Optionally, the corresponding relationship between the ratio and the calibration parameter may be a positive linear relationship, for example, the ratio may be directly used as a calibration parameter for calibrating the image.

It can be understood that, when the color mode of the image to be detected is the color mode of multiple color channels, when the calibration parameters are generated based on the ratio, it is necessary to separately calculate the difference between the average color channel parameters of the two images under different color channels. The ratio determines the calibration parameters for the different color channels.

作为图像在相应的R颜色通道下的校准参数，用于对R颜色通道下的颜色通道参数进行校准，同理，可以将G颜色通道对应的比值

作为图像在相应的G颜色通道下的校准参数以及将B颜色通道对应的比值

作为图像在相应的B颜色通道下的校准参数。Taking the image in RGB color mode as an example, you can directly convert the ratio corresponding to the R color channel

As the calibration parameter of the image under the corresponding R color channel, it is used to calibrate the color channel parameters under the R color channel. Similarly, the ratio corresponding to the G color channel can be

As the calibration parameter of the image under the corresponding G color channel and the ratio corresponding to the B color channel

As the calibration parameter of the image under the corresponding B color channel.

Please continue to refer to FIG. 4 , in step S430, based on the calibration parameters, calibration processing is performed on the color channel parameters of all pixels included in the image to be detected, to obtain a calibrated image to be detected.

In one embodiment, after generating calibration parameters for performing calibration processing on the image to be detected, calibration processing may be performed on the color channel parameters of all pixels included in the image to be detected according to the calibration parameters corresponding to the image to be detected, to obtain a calibrated image data set.

Specifically, the calibration process is performed according to the calibration parameters of the image to be detected under each color channel and the color channel parameters of each pixel of the image to be detected under the corresponding color channel. The calibration processing method may specifically be calculating the product of the calibration parameter of the image under each color channel and the color channel parameter of the pixel of the image under the corresponding color channel, and the product is used as the calibrated color channel parameter, and then realize Calibrate the color channel parameters of each pixel of the image under the corresponding color channel.

After the color channel parameters of all the pixels included in the image are calibrated, the calibrated image to be detected can be obtained.

It can be seen from the above that, based on the color channel parameters of each image in the image dataset, the images in the image dataset are clustered to obtain multiple image cluster sets, and the target with the largest number of images is selected from the multiple image cluster sets. Image clustering set, and based on the color channel parameters of each image in the target image clustering set, calculate the average color channel parameters of the images in the target image clustering set, and then based on the average color channel parameters of the images in the target image clustering set. The color channel parameters are calibrated to obtain the calibrated image to be inspected. First, the image to be inspected is preprocessed for image calibration, and then the image defect location model is used to perform defect location detection according to the calibrated image to be inspected. Compared with The image to be inspected without data preprocessing without image calibration is directly input into the image defect localization model, so that when the image to be inspected is an LCD panel image gathered in a specific color mode, the image defect localization model can also perform accurate defect localization, The accuracy of defect localization by the image defect localization model is improved.

Referring to FIG. 6 , FIG. 6 shows a flowchart of an image processing method according to an embodiment of the present application. The image processing method in the embodiment of the present application may further include steps S610 to S620 , which are described in detail as follows.

In step S610, the calibrated image to be detected is input into the pre-trained image defect localization model.

In one embodiment, when analyzing the color defect of the liquid crystal panel, the image of the liquid crystal panel can be input into the image defect localization model obtained by training, and the image defect localization model can be used to realize the defect localization of the liquid crystal panel image. The localization model is obtained by training the machine learning model. The machine learning model may be a CNN (Convolutional Neural Network, convolutional neural network) model, or a deep neural network model, or the like. The pre-trained image defect localization model can perform defect localization processing on the input calibrated image to be inspected, so as to detect the defect position in the image to be inspected.

In step S620, a defect localization process is performed on the calibrated image to be detected by using the image defect localization model, and a defect localization result is output.

In one embodiment, a defect location process is performed on the calibrated image to be inspected by an image defect location model, and a defect location result is output, and the defect location result is the defect location obtained by performing the defect location process on the calibrated image to be inspected. information.

In the technical solution of the embodiment shown in FIG. 6, a calibrated image to be inspected is obtained by calibrating the color channel parameters of the image to be inspected. Defect localization detection is performed according to the calibrated image to be inspected, compared to directly inputting the image to be inspected without image calibration data preprocessing into the image defect localization model, so that when the image to be inspected is a liquid crystal aggregated in a specific color mode When the panel image is used, the image defect location model can also perform accurate defect location, which improves the accuracy of the image defect location model for defect location.

Referring to FIG. 7 , FIG. 7 shows a flowchart of an image processing method according to an embodiment of the present application. The image processing method in the embodiment of the present application may further include steps S710 to S740 , which are described in detail as follows.

In step S710, a calibration process is performed on the color channel parameters of the images in the image data set based on the average color channel parameters of the images in the target image cluster set to obtain a calibrated image data set.

In one embodiment, when generating the training sample data for training the machine learning model according to the image data set, based on the average color channel parameters of the images in the target image clustering set, the color channel parameters of the images in the image data set can be processed first. The calibration process is performed to obtain a calibrated image dataset. Among them, based on the average color channel parameters of the images in the target image cluster set, the method of calibrating the color channel parameters of each image in the image data set is based on the average color channel parameters of the images in the target image cluster set. The color channel parameters are calibrated in the same way, and will not be repeated here.

In step S720, image defect labeling processing is performed on each image in the calibrated image data set to obtain a labelled image data set.

In one embodiment, when the image defect localization model is trained according to the calibrated image data set, image defect labeling processing may also be performed on each image in the calibrated image data set, that is, the actual defects of each image are preliminarily identified. Location is marked.

In step S730, training sample data is generated based on the labeled image data set.

In one embodiment, after labeling the calibrated image data set to obtain the labelled image data set, training sample data for training the machine learning model may be generated according to the labelled image data set.

Referring to FIG. 8, FIG. 8 shows a specific flowchart of step S720 of an image processing method according to an embodiment of the present application. Step S720 in this embodiment of the present application may include steps S810 to S830, which are described in detail as follows.

In step S810, a target image is selected from the marked image data set.

In one embodiment, after annotating the calibrated image dataset to obtain the annotated image dataset, in order to improve the effect of the machine learning model, some images may also be selected from the annotated image dataset as The target image for data augmentation processing.

Optionally, part of the images may be directly extracted from the labeled image dataset at random, as the target images for data enhancement processing.

Optionally, referring to FIG. 9 , FIG. 9 shows a specific flowchart of step S810 of an image processing method according to an embodiment of the present application. Step S810 in this embodiment of the present application may include steps S910 to S930. Described as follows.

In step S910, a data enhancement probability threshold is obtained.

In step S920, random numbers are assigned to the images in the marked image dataset.

In step S930, the image whose assigned random number is less than or equal to the data enhancement probability threshold is determined as the target image.

In one embodiment, when a target image for data enhancement processing is selected from the marked image data set, in order to ensure randomness, a data enhancement probability threshold for image selection may be preset, and the data enhancement probability threshold may be preset according to the data enhancement probability threshold. For image selection, the data enhancement probability threshold may be a preset probability parameter, for example, a certain probability parameter between 0.5 and 0.8.

Specifically, a random number may be allocated to each image in the marked image dataset in advance, and the random number is generally a parameter between 0 and 1.

The random number assigned to each image is compared with the data enhancement probability threshold, and the image whose assigned random number is less than or equal to the data enhancement probability threshold is determined as the target image, thereby realizing that the image data can be randomly selected from the labeled image data. Select the target images that need to be augmented centrally.

Please continue to refer to FIG. 8 , in step S820 , data enhancement processing is performed on the target image to obtain a processed image.

In one embodiment, after acquiring the target image, data enhancement processing may be performed on the selected target image to obtain a processed image.

Specifically, for any selected target image, various types of data enhancement processing such as color transformation, rotation, scaling and adding noise can be performed on the target image. It can be understood that, for each type of data enhancement processing, it can be determined according to a certain probability whether each target image should perform the data enhancement processing corresponding to the type, so each target image may perform one or more types of data enhancement processing at the same time. Data augmentation processing.

In step S830, training sample data is generated based on the processed image and the labeled image data set.

The training sample data for training the machine learning model is composed based on the image after data enhancement processing and the original image without data enhancement processing.

In the technical solution of the embodiment shown in FIG. 8 , the training sample data for training the machine learning model is jointly generated by using the images that have not undergone data enhancement processing and the images that have undergone data enhancement processing, which can effectively improve the image defects obtained from training. The generalization ability of the localization model improves the performance of the image defect localization model for image defect localization.

Please also continue to refer to FIG. 7 , in step S740, the machine learning model is trained based on the generated training sample data to obtain an image defect localization model.

In one embodiment, the machine learning model is trained based on the generated training sample data to obtain an image defect localization model, and the process of training the machine learning model is to adjust various coefficients in the network structure corresponding to the machine learning model, so that for The input image to be detected is subjected to various coefficient operations in the network structure corresponding to the machine learning model, and the output result is the determined defect location information.

In the technical solution of the embodiment shown in FIG. 7 , when the image to be detected is a liquid crystal panel image gathered in a specific color mode, data preprocessing for color calibration is performed on each image to be detected in the image data set, so that the image and the calibrated image are preprocessed. The color difference between the images in the image data set is reduced, so that the image defect localization model trained on the calibrated image data set images can achieve accurate defect localization for the LCD panel images gathered in a specific color mode, improve the The accuracy of the image defect localization model for defect localization is improved.

The apparatus embodiments of the present application are introduced below, which can be used to execute the image processing methods in the above-mentioned embodiments of the present application. For details not disclosed in the embodiments of the apparatus of the present application, please refer to the above-mentioned embodiments of the image processing method of the present application.

FIG. 10 shows a block diagram of an image processing apparatus according to an embodiment of the present application.

Referring to FIG. 10 , an image processing apparatus 1000 according to an embodiment of the present application includes: a clustering unit 1010 , a computing unit 1020 and a first calibration unit 1030 . The clustering unit 1010 is configured to perform clustering processing on the images in the image dataset based on the color channel parameters of each image in the image dataset to obtain multiple image cluster sets; the computing unit 1020 is configured to cluster images from multiple images Selecting the target image cluster set with the largest number of images in the cluster set, and calculating the average color channel parameter of the images in the target image cluster set based on the color channel parameters of each image in the target image cluster set; the first calibration unit 1030 is used to calculate the average color channel parameter of the images in the target image cluster set based on the target image The average color channel parameters of the images in the clustering set are calibrated for the color channel parameters of the images to be detected, and the calibrated images to be detected are obtained.

In some embodiments of the present application, based on the foregoing solution, the clustering unit 1010 is configured to: based on the color channel parameters corresponding to all pixels included in each image in the image data set, respectively calculate the corresponding to each image in the image data set. Average color channel parameters: Based on the average color channel parameters corresponding to each image in the image data set, the images in the image data set are clustered to obtain multiple image cluster sets.

In some embodiments of the present application, based on the foregoing solution, the clustering unit 1010 is configured to: based on the color channel parameters of all pixels included in each image in the image data set under the respective color channels, respectively calculate each color channel in the image data set. The average color channel parameters of the image under each color channel.

In some embodiments of the present application, based on the aforementioned solution, the first calibration unit 1030 is configured to: determine the average color channel parameter corresponding to the image to be detected based on the color channel parameters of all pixels included in the image to be detected; The average color channel parameters of the images in the image clustering set and the average color channel parameters corresponding to the images to be detected are used to generate calibration parameters; based on the calibration parameters, the color channel parameters of all the pixels included in the to-be-detected images are calibrated to obtain a calibrated image to be detected.

In some embodiments of the present application, based on the foregoing solution, the first calibration unit 1030 is configured to: calculate the ratio between the average color channel parameter of the images in the target image cluster set and the average color channel parameter corresponding to the image to be detected; Based on the ratio, calibration parameters are generated.

In some embodiments of the present application, based on the foregoing solution, the image processing apparatus further includes: an input unit for inputting the calibrated to-be-detected image into a pre-trained image defect localization model; a localization unit for passing the image defect The localization model performs defect localization processing on the calibrated image to be inspected, and outputs the defect localization result.

In some embodiments of the present application, based on the foregoing solution, the image processing apparatus further includes: a second calibration unit, configured to perform a calibration on the color channel parameters of the images in the image data set based on the average color channel parameters of the images in the target image cluster set Perform calibration processing to obtain a calibrated image data set; a labeling unit is used to perform image defect labeling processing on the images in the calibrated image data set to obtain a labelled image data set; a generation unit is used to base on the labelled image. The data set is used to generate training sample data; the training unit is used to train the machine learning model based on the generated training sample data to obtain an image defect localization model.

In some embodiments of the present application, based on the foregoing solution, the generating unit is configured to: select a target image from the marked image data set; perform data enhancement processing on the target image to obtain a processed image; based on the processed image And the labeled image dataset to generate training sample data.

In some embodiments of the present application, based on the foregoing solution, the generating unit is configured to: obtain a data enhancement probability threshold; assign random numbers to the images in the labeled image dataset; assign the assigned random numbers less than or equal to the data enhancement probability threshold The image is determined as the target image.

FIG. 11 shows a schematic structural diagram of a computer system suitable for implementing the electronic device according to the embodiment of the present application.

It should be noted that the computer system 1100 of the electronic device shown in FIG. 11 is only an example, and should not impose any limitations on the functions and scope of use of the embodiments of the present application.

As shown in FIG. 11 , the computer system 1110 includes a Central Processing Unit (CPU) 1101, which can be loaded into a random device according to a program stored in a Read-Only Memory (ROM) 1102 or from a storage part 1108 A program in a random access memory (RAM) 1103 is accessed to perform various appropriate actions and processes, for example, the methods described in the above embodiments are performed. In the RAM 1103, various programs and data necessary for system operation are also stored. The CPU 1101 , the ROM 1102 , and the RAM 1103 are connected to each other through a bus 1104 . An Input/Output (I/O) interface 1105 is also connected to the bus 1104 .

The following components are connected to the I/O interface 1105: an input section 1106 including a keyboard, a mouse, etc.; an output section 1107 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc. ; a storage part 1108 including a hard disk and the like; and a communication part 1109 including a network interface card such as a LAN (Local Area Network) card, a modem, and the like. The communication section 1109 performs communication processing via a network such as the Internet. Drivers 1110 are also connected to I/O interface 1105 as needed. A removable medium 1111, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is mounted on the drive 1110 as needed so that a computer program read therefrom is installed into the storage section 1108 as needed.

In particular, according to embodiments of the present application, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising a computer program for performing the method illustrated in the flowchart. In such an embodiment, the computer program may be downloaded and installed from the network via the communication portion 1109, and/or installed from the removable medium 1111. When the computer program is executed by the central processing unit (CPU) 1101, various functions defined in the system of the present application are executed.

It should be noted that the computer-readable medium shown in the embodiments of the present application may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above. More specific examples of computer readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable Erasable Programmable Read Only Memory (EPROM), flash memory, optical fiber, portable Compact Disc Read-Only Memory (CD-ROM), optical storage device, magnetic storage device, or any suitable of the above The combination. In this application, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program therein. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device . A computer program embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. Wherein, each block in the flowchart or block diagram may represent a module, program segment, or part of code, and the above-mentioned module, program segment, or part of code contains one or more executables for realizing the specified logical function instruction. It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams or flowchart illustrations, and combinations of blocks in the block diagrams or flowchart illustrations, can be implemented in special purpose hardware-based systems that perform the specified functions or operations, or can be implemented using A combination of dedicated hardware and computer instructions is implemented.

The units involved in the embodiments of the present application may be implemented in software or hardware, and the described units may also be provided in a processor. Among them, the names of these units do not constitute a limitation on the unit itself under certain circumstances.

As another aspect, the present application also provides a computer-readable medium. The computer-readable medium may be included in the electronic device described in the above embodiments; it may also exist alone without being assembled into the electronic device. middle. The above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by an electronic device, enables the electronic device to implement the methods described in the above-mentioned embodiments.

It should be noted that although several modules or units of the apparatus for action performance are mentioned in the above detailed description, this division is not mandatory. Indeed, according to embodiments of the present application, the features and functions of two or more modules or units described above may be embodied in one module or unit. Conversely, the features and functions of one module or unit described above may be further divided into multiple modules or units to be embodied.

From the description of the above embodiments, those skilled in the art can easily understand that the exemplary embodiments described herein may be implemented by software, or may be implemented by software combined with necessary hardware. Therefore, the technical solutions according to the embodiments of the present application may be embodied in the form of software products, and the software products may be stored in a non-volatile storage medium (which may be CD-ROM, U disk, mobile hard disk, etc.) or on the network , which includes several instructions to cause a computing device (which may be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present application.

Other embodiments of the present application will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses or adaptations of this application that follow the general principles of this application and include common knowledge or conventional techniques in the technical field not disclosed in this application .

It is to be understood that the present application is not limited to the precise structures described above and illustrated in the accompanying drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims (10)

1. an image processing method, is characterized in that, comprises: Based on the color channel parameters of each image in the image data set, the images in the image data set are clustered to obtain a plurality of image cluster sets; Select the target image cluster set with the largest number of images from the plurality of image cluster sets, and calculate the average color channel parameter of the images in the target image cluster set based on the color channel parameters of each image in the target image cluster set; Based on the average color channel parameters of the images in the target image cluster set, the color channel parameters of the image to be detected are calibrated to obtain a calibrated image to be detected. 2 . The image processing method according to claim 1 , wherein the images in the image data set are clustered based on the color channel parameters of each image in the image data set to obtain a plurality of image clustering sets. 3 . ,include: Based on the color channel parameters corresponding to all pixels included in each image in the image data set, calculate the average color channel parameters corresponding to each image in the image data set respectively; Based on the average color channel parameter corresponding to each image in the image data set, the images in the image data set are clustered to obtain a plurality of image cluster sets. 3. The image processing method according to claim 2, characterized in that, calculating the phase of each image in the image dataset separately based on the color channel parameters corresponding to all pixels included in each image in the image dataset. Corresponding average color channel parameters, including: Based on the color channel parameters of all the pixels included in each image in the image dataset under respective color channels, the average color channel parameters of each image under respective color channels in the image dataset are calculated respectively. 4. The image processing method according to claim 1, characterized in that, based on the average color channel parameter of the image in the target image clustering set, the color channel parameter of the image to be detected is calibrated to obtain the calibrated to-be-detected image. Detect images, including: Determine the average color channel parameter corresponding to the image to be detected based on the color channel parameters of all pixels included in the image to be detected; generating calibration parameters based on the average color channel parameters of the images in the target image cluster set and the average color channel parameters corresponding to the images to be detected; Based on the calibration parameters, calibration processing is performed on the color channel parameters of all pixels included in the image to be detected, to obtain a calibrated image to be detected. 5. The image processing method according to claim 4, wherein the calibration parameter is generated based on the average color channel parameter of the image in the target image cluster set and the average color channel parameter corresponding to the to-be-detected image ,include: calculating the ratio between the average color channel parameter of the images in the target image cluster set and the average color channel parameter corresponding to the to-be-detected image; Based on the ratios, calibration parameters are generated. 6. The image processing method according to any one of claims 1-5, wherein the image processing method further comprises: Inputting the calibrated image to be detected into a pre-trained image defect localization model; Defect localization processing is performed on the calibrated image to be detected by the image defect localization model, and a defect localization result is output. 7. The image processing method according to claim 6, wherein the image processing method further comprises: Based on the average color channel parameters of the images in the target image cluster set, a calibration process is performed on the color channel parameters of the images in the image data set to obtain a calibrated image data set; Perform image defect labeling processing on each image in the calibrated image data set to obtain a labelled image data set; Generate training sample data based on the labeled image data set; The machine learning model is trained based on the generated training sample data to obtain the image defect localization model. 8. The image processing method according to claim 7, wherein, generating training sample data based on the labeled image data set, comprising: From the labeled image data set, select a target image; performing data enhancement processing on the target image to obtain a processed image; Based on the processed image and the labeled image dataset, training sample data is generated. 9. The image processing method according to claim 8, wherein the selection of a target image from the labeled image data set comprises: Get the data augmentation probability threshold; assigning random numbers to images in the labeled image dataset; An image whose assigned random number is less than or equal to the data enhancement probability threshold is determined as the target image. 10. An image processing device, comprising: a clustering unit, configured to perform clustering processing on the images in the image data set based on the color channel parameters of each image in the image data set to obtain a plurality of image cluster sets; The computing unit is used to select the target image cluster set with the largest number of images from the plurality of image cluster sets, and based on the color channel parameters of each image in the target image cluster set, calculate the number of images in the target image cluster set. Average color channel parameters; The first calibration unit is configured to perform calibration processing on the color channel parameters of the image to be detected based on the average color channel parameters of the images in the target image cluster set to obtain a calibrated image to be detected.

Images