JP2004239870A - Spatial filter, spatial filter creation method, spatial filter creation program, screen defect inspection method and apparatus - Google Patents

Abstract

A spatial filter having improved detection power by allowing a filter configuration value to be automatically calculated by using a genetic algorithm in accordance with the state of a screen defect to be inspected, thereby improving detection power, and a method of forming the spatial filter. An automatic calculation program for creating the spatial filter, and a screen defect inspection method and an inspection apparatus using the spatial filter are provided.
A real number filter is created using a genetic algorithm. Further, a screen to be inspected is imaged by the CCD camera 6, and a difference between the image captured by the imaging and the background image 14 created in advance is created to create an inspection image 15. Is used to perform filter processing for defect enhancement, statistically calculate luminance data of each pixel in the image 16 after the filter processing, determine a threshold based on the luminance statistical data, and extract defect candidates.
[Selection diagram] Fig. 1

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a spatial filter used for image processing, and in particular, to a spatial filter created using a genetic algorithm (Genetic Algorithm), a method for creating the spatial filter, an automatic calculation program for creating the spatial filter, and a space for the spatial filter The present invention relates to a method and an apparatus for inspecting a screen defect such as a spot defect using a filter.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, when a defect of a screen of a liquid crystal display device or the like is detected by image processing, a filtering process using a spatial filter is performed to emphasize the feature of the defect. In the filtering process, numerical values constituting the spatial filter and the shape of the spatial filter are devised in accordance with the type of defect (linear, point-like, planar defect, etc.) to be detected, the size, and the like. Then, using the spatial filter created in this manner, a portion having a defect is selectively extracted from the image to be inspected to improve the detection accuracy (for example, see Patent Document 1).
[0003]
[Patent Document 1]
JP 2001-28059 A (paragraphs [0017] to [0021], FIGS. 2 to 6)
[0004]
[Problems to be solved by the invention]
When filter processing is used for defect detection using image processing, it is necessary to adjust the configuration values and shape of the spatial filter so that defects can be detected efficiently and selectively. It is based on an existing spatial filter, and as shown in Patent Document 1, the filter configuration value is an integer, so there is a limit in adjustment work, and it is difficult to increase the detection power. was there.
Also, the adjustment of the filter configuration numerical value has been performed by humans because there was no method for automatically calculating the spatial filter according to the defect. Therefore, there is a problem in that the burden on the creator of the spatial filter is large, and it takes much time to create the spatial filter.
[0005]
The present invention has been made in view of the above-described problems, and enables fine adjustment by automatically calculating filter configuration values using a genetic algorithm in accordance with the state of a screen defect to be inspected. To provide a spatial filter with improved detection power, a method of producing the spatial filter, an automatic calculation program for producing the spatial filter, and a method and an apparatus for inspecting a screen defect using the spatial filter. I have.
[0006]
[Means for Solving the Problems]
A spatial filter according to the present invention is a spatial filter for image processing created using a genetic algorithm, wherein the filter numerical values constituting the spatial filter are real numbers.
The spatial filter (hereinafter, also simply referred to as “filter”) of the present invention is a filter having a real number which is automatically calculated based on a genetic algorithm and has a real number. It is possible to make an immediate response and fine adjustment is possible, so that the detection power can be improved.
[0007]
Further, in the spatial filter of the present invention, the filter numerical values are arranged symmetrically with respect to the filter center.
Therefore, it is only necessary to arrange the same numerical value in a copy point-symmetrically with respect to the center of the filter, so that the data amount can be substantially reduced and the time required for automatic calculation of the filter can be reduced.
[0008]
Further, by arranging the filter numerical value in a substantially circular shape with respect to the center of the filter, the filter is not affected by the luminance change near the four corners generated by the normal rectangular filter, and the defect size is large. However, accurate defect detection becomes possible. Further, since the data amount is further reduced, the automatic calculation time of the filter can be further reduced.
[0009]
Furthermore, by setting the data to be put in an area substantially equidistant from the center of the filter to the same numerical value, a substantially circular filter can be formed, and the data amount is further reduced. The time can be significantly reduced.
[0010]
Further, a method of creating a spatial filter according to the present invention is a method of creating a spatial filter for image processing based on a genetic algorithm,
In an input image including a detection target, a detection region including a defective portion to be detected by a spatial filter, and a step of setting a plurality of partitioned detection regions of a region including a normal portion that may not be detected,
Encoding the data of the spatial filter and expanding it into one-dimensional data, thereby setting the genetic algorithm as a gene;
A step of filling the numerical value in the gene by generating a real random number,
Preparing a large number of said genes in combination of real numbers,
Performing a crossover process and / or a mutation process on the prepared gene to repeat generation alternation while generating a new gene;
Returning a new gene generated by the crossover process and / or the mutation process to the spatial filter by a procedure reverse to the encoding, and performing a filter process on the image of the partitioned detection region;
A gene pattern is obtained such that the difference between the calculation result of the detection area of the defective part in the filtering process and the absolute value of the calculation result of the detection area of the normal part becomes maximum and the calculation result in the flat part of the image becomes zero. And a process.
[0011]
If a genetic algorithm is used, crossover processing and mutation processing are performed on a large number of prepared genes, and generation alternation is repeated while constructing a new gene. An optimal solution or a suboptimal solution close to the optimal solution can be obtained. That is, in order to create and verify a filter, an input image including a detection target is used, a plurality of divided detection areas are set in the image, and filter data set in two dimensions is encoded and primary-coded. By expanding to the original, it is set as a gene of the genetic algorithm, and furthermore, the numerical value in the gene is filled by generating a real random number, a large number of these genes are prepared, and crossover processing is performed using the prepared genes. And / or a new gene generated by performing the mutation process is returned to the filter in the reverse procedure of the above-described coding, and the image of the detection region is subjected to the filter process. And a gene pattern in which the difference between the absolute value of the calculation result of the normal part and the calculation result of the normal part becomes maximum, and the calculation result in the flat part of the image becomes zero. This makes it possible to automatically create a filter consisting of an optimum real number for detecting the set detection target.
[0012]
In addition, the spatial filter creation program according to the present invention includes, in an input image including a detection target, a plurality of detection regions including a defective portion to be detected by a spatial filter and a region including a normal portion that need not be detected. A function to set a divided detection area,
By encoding the data of the spatial filter and developing it into one-dimensional data, a function of setting as a gene of a genetic algorithm,
A function to fill the numerical value in the gene by generating a real random number,
A function of storing a large number of the genes in a combination of real numbers and performing a crossover process and / or a mutation process on the stored genes to repeat generation alternation while generating a new gene. When,
A function of returning a new gene generated by the crossover process and / or the mutation process to the spatial filter by a procedure reverse to the encoding, and performing a filter process on the image of the partitioned detection region;
A gene pattern is obtained such that the difference between the calculation result of the detection area of the defective part in the filtering process and the absolute value of the calculation result of the detection area of the normal part becomes maximum and the calculation result in the flat part of the image becomes zero. The feature is that a computer realizes the functions.
[0013]
Therefore, the spatial filter of the present invention can be automatically created by the automatic calculation program based on the genetic algorithm configured as described above, and the space having a high power of detection corresponding to the state of the defect to be detected. A filter can be created in a short time.
In addition, when performing the automatic calculation by this program, the upper limit and the lower limit of the filter numerical value are set in advance. Within the range, an optimum filter having a real number is automatically calculated.
[0014]
As the input image, an image from which an actual defect is to be detected is used, or an artificially created image having a defective portion whose luminance difference and / or size is changed stepwise is used.
When using artificial images that artificially create defects, it is possible to consider various defect patterns rather than using actual images, and to create and verify filters to some extent in a state corresponding to the actual case Can be done in a short time.
[0015]
According to the screen defect inspection method of the present invention, in the filter processing of image processing for detecting a screen defect to be inspected, the spatial filter according to any one of claims 1 to 4 or the creation according to claim 5. A spatial filter created by the method or a spatial filter created by the creation program according to claim 6 is used.
Therefore, the filter processing is performed using a filter having a real number, so that fine adjustment can be performed in accordance with the state of the defect, and the detection power can be improved.
[0016]
Further, the screen defect detection method of the present invention includes a step of imaging a screen to be inspected, a step of performing filter processing for defect enhancement on an image captured by the imaging, and Extracting a defect candidate by setting a threshold value for the
The spatial filter according to any one of claims 1 to 4, or the spatial filter created by the creating method according to claim 5, or the spatial filter created by the creating program according to claim 6. Is used.
[0017]
According to another aspect of the present invention, there is provided a screen defect inspection apparatus including: an imaging unit configured to capture an image of a screen to be inspected; and a computer that incorporates the spatial filter creation program according to claim 6. Is configured to execute the screen defect inspection method according to claim 7 or 8.
The spatial filter creation program based on the genetic algorithm and the inspection program for executing the defect detection processing may be mounted on the same computer as the main body of the inspection apparatus, or may be mounted on another computer.
In the case of realization by a separate computer system, the spatial filter of the present invention is created by a personal computer for creating a spatial filter, and the computer on the inspection apparatus side performs the filtering process using the spatial filter.
[0018]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
Embodiment 1 FIG.
A. Processing flow of genetic algorithm
First, the processing flow of the genetic algorithm applied in the present invention will be described with reference to the flowchart of FIG. The description from A (1) to A (5) is an example of a genetic algorithm, and is not limited to this processing in the present embodiment.
(1) Generation of initial solution group (step S1)
A genetic algorithm (hereinafter sometimes abbreviated as “GA”) expresses a solution in the form of a gene when solving a problem. FIG. 5 shows an image of a solution expressed in the form of a gene. Although two values of 0 and 1 are entered as numerical values constituting the solution, the values are not limited to these values, and values are entered according to the problem. In some cases, characters may be entered instead of numbers.
If the solution cannot be replaced with the form of a gene, GA cannot be applied, so for a problem that cannot be replaced as it is, the problem must be reinterpreted and the solution must be expanded to the form of a gene. . At this time, even if the solution of the problem and the shape of the gene are not the same, there is no problem as long as they are converted to correspond one-to-one.
If the solution can be replaced with the form of a gene, several tens to several hundreds of solutions represented in the form of a gene are prepared as an initial solution group. The number of solutions to be prepared may be larger depending on the situation. In the embodiment, 1,000 initial solutions are prepared.
In the prepared solution, values are randomly distributed within a possible range of the solution so as to minimize bias. Depending on the problem to be applied, there may be a pattern (lethal gene) for which a solution does not hold, but the evaluation value described in the following (2) should be set to 0 (the lowest value), or the gene should be replaced at that time. The problem is dealt with by erasing from the solution group.
When the form of the solution gene is applied to an actual problem, it may be realized in any form as long as it is represented by a one-dimensional data string as shown in FIG. When a program is formed, it is often represented by an array, and in the embodiment, an array (for example, an array composed of 64-bit floating point numbers. Since real numbers include integers, data types are particularly limited. However, an array composed of 32-bit integer gutter data may be used.)
[0019]
(2) Solution evaluation value calculation (steps S2 and S6)
Next, an evaluation function is set so that the generated solution can be evaluated. The evaluation function is used to calculate an index (evaluation value) that indicates how close the optimal solution is when the solution expressed in the form of a gene is returned to the actual solution (here, the filter configuration). Yes, settings are made so that the closer to the optimal solution, the greater the value that is output. The evaluation function is not particularly limited to numerical calculation, and may be obtained as a result of an algorithm. In the embodiment, the evaluation function is set based on the process of returning the solution expressed in the form of the gene to the form of the filter configuration and performing the filtering process on the detection region, and becomes 0 in a flat portion of the image. The larger the difference between the processing result for the defective part and the absolute value of the processing result for the normal part, the better the defective part is detected, and the better the solution is. Is an evaluation function.
The GA uses the evaluation value obtained as a result of the evaluation function to determine whether the solution is superior, and performs processing so as to leave an excellent solution.
[0020]
(3) Selecting a solution to be processed (step S3)
When the solution is replaced with the gene form and the evaluation function can be set, the solution is crossed over and mutated. However, the crossover process uses two solutions. Two solutions must be selected from the solution group.
In GA, processing is performed on the assumption that a superior solution is generated from a superior solution. Therefore, a solution is selected from a parent solution group with a high evaluation value (excellent solution) if the evaluation value is high. If it is low (recessive solution), a solution is selected by performing an operation so as to be selected with a low probability.
By doing so, the probability of using a solution having a low evaluation value is reduced, and the solution is naturally eliminated.
[0021]
(4) Crossover and mutation processing (steps S4 and S5)
As shown in FIGS. 6 and 7, the selected solution is subjected to crossover processing (crossing) and mutation processing to generate a new solution.
[0022]
[Crossover]
The solution expressed in the form of a gene is cut at an arbitrary position, and exchanged between two solutions (parent) to generate a new solution (child). In this process, a solution with a different pattern from the past is generated (the same solution may be generated in the process near convergence, but this is not a problem).
FIG. 6 shows an example of the crossover process. There are several types of crossover methods, and FIG. 6 shows a method of cutting one point (one-point crossover). In addition, there are a method of cutting at two points (two-point crossover) and a method of randomly selecting a crossing pattern, and these methods may be used.
As a result of performing the crossover processing, a pattern (lethal gene) that does not exist as a solution may occur. In this case, if the evaluation value is 0 (the lowest value) and the evaluation value cannot be used, Control crossover to avoid lethal genes.
[0023]
[mutation]
A part of the solution is randomly selected and replaced with other data. In FIG. 7, one data is replaced. However, in some cases, several data may be changed at the same time.
By performing this operation, even if a local solution falls in the middle of the process, it is possible to escape from the local solution and further converge to the optimal solution.
Even in the mutation process, a lethal gene may be generated. Therefore, as in the case of the crossover process, control is performed by using an evaluation value, or the data is generated so that the gene does not become a lethal gene.
[0024]
(5) End determination (step S7)
While performing crossover processing and mutation processing, retain the evaluation value with the highest evaluation value for each generation, and set a value that suits the purpose (optimal solution) or a value that does not cause any problem as a solution ( When the (sub-optimal solution) occurs, the process ends. If the level at which the solution is not reached has been reached, the process returns to the solution selection process in step S3, and the GA process is repeated.
Depending on the problem to be applied, it may not be clear whether the solution being calculated is close to the optimal solution, but in this case, the process is forcibly terminated by the number of repetitions, The level of the solution is determined while observing the change in the maximum evaluation value (for example, when the rising curve of the maximum evaluation value for each generation change has settled substantially horizontally), and the processing is terminated.
[0025]
B. Application of Genetic Algorithm to Automatic Filter Calculation Program
(1) Gene expression of filter
To apply a genetic algorithm to a filter, the solution must first be represented in the form of a gene. In the present invention, since the filter data (numerical values or coefficients constituting the filter) is a solution, this is expressed in the form of a gene. Filter data used in image processing is two-dimensional data.
FIG. 8 shows an example of expanding the filter data of the 5 × 5 pixel filter into one-dimensional data. This figure is an example in which a two-dimensional array is decomposed for each row and recombined in a one-dimensional form to represent a genotype. There is no problem with this method as long as the rules are not changed in the same process. For example, the data may be expanded to one-dimensional data by another method, such as expansion for each column.
[0026]
Once the shape of the gene is determined, fill it with numerical values. At this time, the values are filled using random numbers so that the values are dispersed. In the embodiment, a real random number is generated and used.
Here, how to set the range of the upper limit and the lower limit which the filter numerical value can take becomes a problem. In the embodiment, the upper limit is set to 10 and the lower limit is set to -10, and a random number is generated within the range to generate a solution.
The upper limit and the lower limit are not limited to these values. In actual image processing, the range of data that can be taken by the filter numerical value only acts as a bias of the detection sensitivity. As long as the condition is fixed, the processing is not affected, and there is no particular problem.
By the way, if the upper and lower limits are not set, the amount of calculation becomes enormous, the processing does not converge, and the detection sensitivity varies, making it impossible to compare whether the solution is excellent.
[0027]
(2) Evaluation function setting for filter solution
Set up an evaluation function to evaluate the solution.
In the filtering process for defect detection,
(A) If the detection target has a defect feature, the processing result is as large as possible.
(B) If there is no flat data, the processing result is 0.
(C) When there is a negative correlation with the feature of the defect, the processing result is negative and the absolute value is as large as possible.
It is desirable that the above three conditions hold.
However, the condition (c) does not need to be considered as long as the condition (b) is satisfied because the relationship is exactly opposite to that of the condition (a), and is excluded in the processing of the embodiment.
In addition, for data that is not flat data but has a change in luminance but is not to be detected as a defect, consideration is given to reducing the processing result.
[0028]
Based on the above conditions, humans judge the part that they want to detect as a defect and the part that they do not want to detect with the filter size designed from the target image to be automatically calculated, and select several points, and use the data to evaluate the evaluation value. Ask for.
In the embodiment, the point using a plurality of areas is a point, and the selection of a plurality of areas has the effect of smoothing, so that the processing can be stabilized at the same time.
In addition, by selecting the same part a plurality of times, it is possible to calculate the part so that it reacts more strongly than the other parts. Is also possible. That is, the difference between the sum of the filter processing results of a plurality of parts to be detected as a defect and the sum of the absolute values of the filter processing results of a plurality of parts not to be detected as a defect is large, and the calculation result in a flat part of the image is 0. Then, an evaluation function is set so that the evaluation value becomes higher when the evaluation value becomes higher, and the evaluation value is obtained.
[0029]
FIG. 9 shows a pseudo sample of a panel defect used for creating and verifying the algorithm.
In order to use the actual sampled defect for algorithm verification, it is necessary to arrange various variations of images so that the defect pattern does not drop as much as possible, but it requires a considerable amount of time In the embodiment, the algorithm is created and verified by using the pseudo stain 20 created by changing the luminance value and the area stepwise.
FIG. 9 shows an area setting image of the program created in the embodiment, in which four portions 21 to be detected and five portions 22 not to be detected are set, that is, a total of nine portions. The selected area is a portion surrounded by a rectangle, and the area where the pseudo-stain 20 is included in the rectangle is the part 21 to be detected, and the other square areas are the parts 22 not to be detected. The image size of each of the selected detection areas 21 and 22 is 35 × 35 pixels.
[0030]
The evaluation value is obtained using the images of the set detection areas 21 and 22. For the calculation of the evaluation value, the convolution operation generally used as the operation of the filter processing is directly used.
FIG. 2 shows a subroutine in the evaluation value calculation process of FIG. The one-dimensional array represented in the form of the solution is returned to the form of the filter, which is two-dimensional data, by performing processing reverse to the processing shown in FIG. 8 (step S21 or S61), and the selected target area 21, A convolution operation is performed on the block 22 (step S22 or S62).
First, an offset value is added to all the filter numerical values so that the result becomes 0 in a portion where there is no change in luminance, and so that the total sum of all the filter numerical values becomes 0. The offset value is obtained by dividing the sum of the filter constituent values by the number of filter constituent values (for example, 225 in the case of a 15 × 15 pixel filter, and in the case of a circular filter to be described later, the number of areas in which the numerical values are stored), and Calculate by inverting.
With respect to the filter to which the offset value is added, a convolution operation is performed on a portion (defective area) 21 to be detected by the filter in the above-described area setting, and the result is added as it is, and the portion not to be detected (normal For the area 22), the absolute value of the result is taken and subtracted, and this calculation is performed for all the set portions to obtain an evaluation value.
[0031]
(3) Crossover processing
FIG. 3 shows a subroutine in the crossover process of FIG.
As described above, in the crossover process, two solutions are selected from a prepared initial solution or a group of solutions replaced after the crossover and mutation processes in FIG. 1 (step S41). Next, the cutting position of the solution is set by a random number (step S42), and a part of the data is exchanged between the parent solutions as shown in FIG. 6 to generate the child solution (step 43).
[0032]
(4) Mutation processing
FIG. 4 shows a subroutine in the mutation process of FIG.
In the mutation process, first, a mutation position (a position at which a part of the data of the parent solution is exchanged) is set by a random number (step S51, see FIG. 7). Then, a part of the parent solution data is exchanged to generate a child solution (step S52), and the generated child solution is exchanged with the parent to generate a new solution (step 53).
As described above, the parent is replaced by the generated new solution (child) while the crossover and mutation processes are repeated, and the evaluation value calculation of step S6 in FIG. 1 is performed on the replaced solution (parent). Thus, the filter numerical value can be made closer to the optimal solution.
[0033]
Next, the gene expression of the above-described filter will be described with examples.
In the first embodiment, gene expression was performed using all data in a two-dimensional space (square area) constituting a filter. By adopting such a configuration, a filter that emphasizes a defect to be detected can be automatically calculated as long as the defect is within the filter size set for calculation.
Even in the case of a linear defect, it may not fit in the length direction, but when the filter is moved in the direction of the line, the characteristics at each position do not change so much, so that the linear defect can be used as it is. However, in this configuration, since the calculation is performed in consideration of all the data, the amount of calculation increases, and there is a disadvantage that the calculation takes time. In addition, since the filter to be calculated is swung to the shape of the defect, the filter becomes limited as the defect is more limited in detection as the shape of the defect moves away from the circle.
When detecting a linear defect, there is no other way than to use such a filter, but for a planar defect (smear, unevenness, etc.), the number of filters applied to the inspection should be as small as possible depending on the directionality. It is necessary to reduce the number and make the structure generally usable.
[0034]
Therefore, in a second embodiment, an example will be described in which data generation is restricted so that the direction dependence of the filter is eliminated, and the calculated result is configured so as to be generally applicable to a planar defect. . Since the base is a genetic algorithm, only the gene expression changes, and the other parts do not change.
The defect to be calculated is a linear defect, which always has a directivity, and therefore has no directivity and has a certain degree of shape, such as a stain defect.
In the configuration of the filter, the following concept and method are applied.
[0035]
(I) Use of symmetry
Among filters used in image processing, filters that detect a certain amount of lump are generally arranged point-symmetrically with respect to the center of the filter as an axis.
When the condition for eliminating the direction dependency is added to this condition, the arrangement of the data is further limited, and the data is symmetrical about four axes, that is, a vertical axis, a horizontal axis, and two diagonal axes. Placed in
FIG. 10 shows an example of data arrangement using symmetry. According to this method, it is only necessary to create the data of the area surrounded by the thick line in FIG. 10 and copy the value to the position of the same code outside the thick line frame in FIG.
Due to this limitation, taking a 9 × 9 pixel filter as an example, in the first embodiment, 81 data must be used for processing. However, by using symmetry, 15 data can be used for processing. Can be reduced.
[0036]
(Ii) Application of circular filter
As the filter size to be designed increases, the diameter in the horizontal direction and the diameter in the oblique direction greatly differ, and thus the following problems occur.
FIG. 11 is a diagram for explaining this, and shows an example of a filter of 9 × 9 pixels. Actually, the filter configuration numerical value is a point value and has no area information, but is represented as a square area for easy understanding in description.
As shown in FIG. 11, there is an extra area in the diagonal direction that is larger than the diameter in the horizontal and vertical directions, and this area is an area that is extraly considered in the filter calculation. For example, if there is a bright point at the position b in FIG. 11, it affects the filter calculation value. However, if there is a bright point at the position a at the same distance from the center, it is out of the range of the filter processing. , Does not affect the filter calculation value. Therefore, the position of b is an irregular area when calculating the filter value.
[0037]
Therefore, in order to solve this problem, the shape of the filter is made closer to a circle, and an extra area is omitted from the processing.
FIG. 12 shows an example of data arrangement in the case where the concept of (i) data symmetry and the concept here are applied to a 9 × 9 pixel filter.
In the image processing, since discrete data is handled, a circular shape is not accurately obtained as shown in FIG. However, rather than performing the processing in a square state as it is, each direction is somewhat improved. In addition, as shown in FIG. 12, by considering the circular filter, 15 data can be further reduced to 13 data, which is advantageous from the viewpoint of data amount.
[0038]
The processing area set in FIG. 12 is set as an area to be processed within an area touching the circle shown in FIG. 11, and the other areas are set as areas to be omitted from the processing.
This definition is not limited, and the area of the area divided by the circle may be compared, and if the area inside the circle is large, the area may be defined as a processing area, or may be defined as a processing area. Although the direction area becomes smaller, an area completely inside the circle may be set as the processing area. Although the circle is considered at the center of the center pixel, the circle may be outside the center pixel.
[0039]
(Iii) Equidistant areas are processed with the same data
In the filter processing, the center of the filter is the position where the result of the filter processing is stored. Therefore, it can be said that the filter configuration numerical value is a coefficient indicating how much a nearby pixel exerts an influence on a central pixel in consideration of a detection target. Then, those that are equidistant from the center pixel should have the same effect, so that those that are equidistant from the center pixel must have the same numerical value.
FIG. 13 shows the result of applying this concept to a filter of 9 × 9 pixels.
Similar to FIG. 12, since discrete data is handled in the image processing, there is no strictly equidistant area in the 1/8 area where data is generated. Therefore, in FIG. 13, the distance from the center is divided every 0.3 steps (pixels), and the values falling within the same step are set to the same numerical value as equidistant data. For example, in FIG. 13, the value of d is 0.3 × (13 to 14) = 3.9 to 4.2 pixels from the center pixel a, and the area in this range has the same numerical value as equidistant data. Is set.
By performing such processing, data can be further reduced from 13 to 10. In FIG. 13, the distance is divided every 0.3 steps, but this value is not particularly limited, and any value of 1 or 0.5 may be fixed in the process.
[0040]
In addition, by adopting such a form, it is possible to weight the filter configuration numerical values based on distance data from the center. For example, it is possible to design a filter configuration numerical value such that the influence of the pixel is larger nearer the center and smaller as the distance from the center is smaller.
For example, FIG. 14A shows the case where the same data range is set in all regions, but as shown in FIG. 14B, the center position of the value of a in FIG. The data is set as described above, and this range is reduced as the distance from the center deviates, and the value of e in FIG. 13, which is the end data, is set in the range of +5 to -5. The data during this time can be distributed by the ratio of distance to set a data range, and can be calculated by a genetic algorithm.
By performing the above operation, the direction dependency of the filter can be eliminated, the amount of calculation of the genetic algorithm can be reduced, and the calculation time can be shortened.
[0041]
FIG. 15 shows an example of a filter value of 27 × 27 pixels automatically calculated based on the genetic algorithm described above. This filter is substantially constituted by a circular filter, and the numerical value constituting the filter is a real number with three digits after the decimal point. Therefore, for a defect such as a stain, the filter pattern can be more finely adjusted by a real number, and the detection power can be improved.
On the other hand, FIG. 16 shows an example in which the same filter value of 27 × 27 pixels is configured by an integer. This was created based on the inventor's experience so far, and the results when the real number filter shown in FIG. 15 and the integer filter shown in FIG. 16 were applied to a certain spot image were as follows.
Real number filter evaluation value 7.39
Integer filter evaluation value 7.32
The evaluation value was obtained by applying a filter process to the stain image and normalizing the result, in order to determine how many times the maximum brightness value of the stain portion becomes the variance σ of the brightness value of the entire screen. Things.
As a result, the detection power using the evaluation value is almost the same. However, in the case of a real number filter, it can be created in tens of seconds due to automatic calculation, whereas in the case of an integer filter, trial and error must be repeated for creation and evaluation, which takes a considerable amount of time.
FIG. 17 shows an example in which the solution does not converge in the genetic algorithm. In this case, it can be seen that the filter numerical value is a meaningless real number.
[0042]
Other embodiments
In the first embodiment, a square filter is set, and in the second embodiment, a filter having a circular area is set. However, the filter may have another shape. For example, in a filter for detecting a line, a rectangle as shown in FIGS. 18A and 18B is conceivable, and there is no problem with other arbitrarily set shapes.
Also, the filter size is not particularly limited, but can be adjusted from 10 × 10 pixels to 55 × 55 pixels so as to correspond to the size of the stain defect.
[0043]
As described above, according to this embodiment, a filter having a real number can be created in a very short time by automatically calculating a filter according to a target defect. In addition, since the constituent values can be finely adjusted according to the size of the defect, the state of the luminance difference, and the like, the detection power for the defect can be improved.
Furthermore, since the filter size and the filter shape are not limited and can be designed freely, it is possible to quickly respond to the detection of various defects.
[0044]
Embodiment 2 FIG.
FIG. 19 is a configuration diagram showing the outline of the screen defect inspection apparatus of the present invention.
This embodiment shows an example of detecting a spot defect on a liquid crystal display screen by using a spatial filter automatically calculated based on the above-described genetic algorithm.
Here, for example, the screen 10 to be inspected is a projection screen of a liquid crystal panel (also referred to as a liquid crystal light valve) 2 using TFT elements by the projector 1. When performing an inspection, the image 4 is projected on the screen 3 by the projector 1. The image 4 is drawn by giving a predetermined pattern to the liquid crystal panel 2 by the pattern generator 5. For example, an image 4 is picked up by a CCD camera 6 as an image pickup means, and the image signal is converted from an analog signal to a digital signal by an A / D converter (not shown) and is taken into a computer 7 which is a main body of the inspection apparatus. At this time, the image data is represented by, for example, 12-bit data with black being “0” and white being “4095” for each pixel by the A / D converter with a luminance value of 4096 gradations. Further, the computer 7 incorporates an automatic calculation program for creating a filter based on a genetic algorithm and an inspection program for detecting defects, and according to these programs, the computer 7 causes the image of the image 4 stored in the image memory to be read. By processing the data by a method described later, a stain defect is detected for each light / dark defect. At the time of defect detection, after performing filter processing for defect enhancement using the filter created by the above automatic calculation program, statistical processing of the luminance information in the detected image is performed, and based on the statistical data, A threshold for extracting the defect candidate is determined, the defect candidate is extracted, and an evaluation value for quantitatively evaluating the extracted defect candidate is calculated. These inspection results are displayed on the display device 8.
[0045]
FIG. 20 is a flowchart used in the stain defect detection processing, and FIG. 21 is a schematic diagram of an image after image processing in each stage from an input image to a stain defect detection image. This spot defect detection processing is automatically performed according to the inspection program incorporated in the computer 7 or the image processing apparatus.
The processing procedure will be described with reference to the flowchart of FIG.
[0046]
(1) Display area extraction processing (step S11)
Extracting a display area means extracting only a screen portion to be inspected from an input image captured by imaging. For example, FIG. 21A shows an input image at the time of imaging. As shown in this figure, the input image 11 at the time of imaging includes the edge portion 31 of the screen 3 or the display area image 12 corresponding to the screen portion is not exactly rectangular but is distorted obliquely with respect to the screen 3. May go out. This is because the screen 3 and the CCD camera 6 of the image pickup means are not exactly parallel, or distortion occurs due to the characteristics of the projection lens of the projector 1 and the lens characteristics of the CCD camera 6. . In addition, even when an image of a liquid crystal panel or the like is directly captured instead of indirectly captured as described above, distortion or deformation of the input image may occur. Of course, when the imaging unit or the screen of the inspection target is accurately set and imaged so that the edge portion 31 does not enter as described above, for example, the field of view is set so that the entire screen portion of the detection target falls within the field of view of the imaging unit. When set correctly, the display area extraction processing and the correction processing described below can be omitted, and the image captured by imaging directly becomes the original image.
[0047]
In the case where the input image 11 is distorted as shown in FIG. 21A, only the display area image 12 of the screen portion is extracted as shown in FIG. A display area correction image 13 that is deformed and corrected to an accurate rectangle is created. Here, the display area correction image 13 is an original image to be actually inspected.
The image correction processing by the geometric deformation detects the coordinates of the four corners of the display area image 12 that does not include the edge portion 31 of the screen 3 by pattern matching processing, and makes the coordinates match the coordinates of the four corners of the rectangle. This is performed by performing coordinate conversion on all pixel data of the display area image 12.
By setting the size of the rectangle set at this time to, for example, 1200 × 1000 pixels, the image size of the original image 13 becomes 1200 × 1000 pixels. This size is not particularly fixed, and may be set to a size equal to or close to the pixel size of the CCD camera.
[0048]
(2) Background image difference processing (step S12)
The background image difference process is a process of subtracting a previously created background image 14 as shown in FIG. 21C from the original image 13 created as described above. By subtracting the background image 14 from the original image 13, other than the defects existing in the liquid crystal panel to be inspected included in the original image 13, the defect-like brightness caused by the illumination of the imaging means, the lens characteristics, etc. Changes can be eliminated. In addition, in this difference processing, 2048 (× １ ／ value of 4096 gradations) is added as an offset value after the difference processing so that the luminance data does not become a negative value. Further, the background image 14 is based on images of a plurality of (for example, about 20) liquid crystal panels with as few defects as possible, which are imaged using the same optical system and the same image pickup system. It is created by averaging the luminance data, is created in advance by the same method as the original image 13, and is stored in the image memory of the computer 7.
As a result of the background image difference processing, a background difference image as shown in FIG. Then, if any, the background difference image (inspection image) 15 includes a spot defect 25 having a different size and contrast.
[0049]
(3) Filter processing (step 13)
In this filter processing, as described above, a spatial filter created by an automatic calculation program based on a genetic algorithm, for example, a filter shown in FIG. 15 is subjected to filter processing.
As a result of the filter processing, a detection image 16 as shown in FIG. That is, since this filter is created in accordance with the size and the contrast of the stain defect 25 to be detected, the filter process can emphasize the stain defect corresponding to the size and the contrast. For a stain defect or other defect portion that does not correspond to the size or contrast, an enhancement process can be performed by separately creating a filter corresponding to the defect according to an automatic calculation program.
Before performing the filtering process, the background difference image (inspection image) 15 is subjected to a noise removal effect as a pre-process for separating a stain defect from other background portions (portions without defects). A certain flattening process may be performed.
[0050]
(4) Statistical calculation (Step S14)
Next, as described above, for the detected image 16 in which the portion (region) considered to be a stain defect 25 or a defect candidate is emphasized, the luminance data of each pixel in the detected image 16 is used to cover the entire screen. Statistical calculation of the luminance data of. In the statistical calculation, the average value Ave and the standard deviation σ of the luminance data are obtained. From these two pieces of luminance statistical data, a threshold value for detecting white spots and black spots is determined as follows, for example.
White spot threshold: Ave + Kσ
Black stain threshold: Ave-Kσ
However, K is an arbitrary numerical value and is usually set to 2 to 4. This numerical value is adjusted according to the calculation result of the filter processing according to the detection target.
Therefore, the threshold value for detecting white spots and black spots can be automatically determined based on the statistical luminance data by statistically calculating the luminance data in the detected image 16. Therefore, the threshold value is not artificial, trial and error, but objective and relative.
[0051]
(5) Extraction of defect candidate (step S15)
Then, it is determined whether there is a white spot defect or a black spot defect in the detected image 16 based on the threshold. That is, the operation result I of the above filter processing is
If there is a region where I> Ave + Kσ, the region is extracted as a white spot defect candidate. Also,
If there is a region where I <Ave−Kσ, the region is extracted as a black spot defect candidate.
When the result I of the filtering process falls within the range between the two threshold values, there is no stain defect, and the product is determined to be non-defective at this stage, and the inspection ends.
[0052]
(6) Blob processing (step S16)
In the above-described extraction processing, when a white spot defect candidate, a black spot defect candidate, or both of the defect candidates are extracted, blob processing is performed on the defect candidate, and the size, coordinate position, A number and a luminance value for identification are obtained. At this time, if the extracted defect candidate is a white spot defect candidate, the maximum value of the luminance value is obtained, and if the extracted defect candidate is a black spot defect candidate, the minimum value of the luminance value is obtained. A blob is a specific grayscale value existing in an image cut out by binarization processing or the like, or a “block” (area) having a range of values. Here, the “blob processing” is processing for obtaining various characteristic values of the area, and various characteristic values in the defect candidate area extracted by using a blob tool or the like used in image processing. To ask for. The characteristic value in the defect candidate area obtained here is used in calculation of an evaluation value described below.
[0053]
(7) Calculation of evaluation value (step S17)
Here, the above-described luminance statistical data (average value Ave, standard deviation σ) and characteristic values (maximum luminance value, minimum luminance value) obtained by the blob processing are used for the stain defect detected as the defect candidate and subjected to the blob processing. Then, the evaluation value E is calculated by the following equation.
White stain:
E = (maximum luminance value-Ave) / σ
Black stain:
E = (Ave−minimum luminance value) / σ
By using these formulas, it is possible to quantitatively evaluate the white spot defect and the black spot defect detected as the defect candidates together with their size (area), coordinate position, and number (Blob number) by objective data. it can. Therefore, the detection accuracy of the stain defect is high. Note that the area of the spot defect can be obtained by counting the number of pixels in a region corresponding to the spot defect. Further, the coordinate position of the spot defect can be obtained from the X and Y coordinates of the barycentric position of the cut shape.
[0054]
In addition, by shifting the above-mentioned threshold value to a non-defective product side to extract a defect candidate, and calculating the evaluation value of the defect candidate, the product is classified into several groups in several stages based on a result of the evaluation value. It becomes possible to rank defects (grades), grade products, and determine defects. For example, when a liquid crystal panel is used as a light valve of a projector for each of R (red), G (green), and B (blue) colors, the relative luminous efficiency is green (when the wavelength is around 555 nm). Since it is the highest, it is the strictest when it is green. For example, the coefficient of the threshold calculation of defect candidate extraction is set to 2, and the products for which no defect candidate is extracted with this threshold are G products, the defect candidates are extracted and all the evaluation results are calculated as evaluation results. Can be classified as an R product if the evaluation value of the defect candidate is less than 3 and a B product if the evaluation value of the defect candidate is 3 or more and a threshold value of failure and less than 4 as a defective product.
Therefore, products can be classified by grade within good products, and grades such as upper, middle and lower can be made within the same grade.
[0055]
In this embodiment, since the spot defect is emphasized by using a filter having a real number which is automatically calculated based on the genetic algorithm as described above and has a real number, the present embodiment is present on a screen of a display device such as a liquid crystal panel. Regardless of the size of the defect and the level of the contrast, the stain defect can be automatically detected with high accuracy, and the stain defect can be individually and quantitatively evaluated.
In addition, since spot defects are evaluated based on luminance statistical data, collecting and analyzing the quality data of products and parts can be used for quality control, and a method aimed at further improving quality is developed. It is also possible to build.
[0056]
In this embodiment, the automatic calculation program for creating a filter based on a genetic algorithm and the inspection program for detecting a defect have been described as being incorporated in the same computer (inspection apparatus main body) 7. It may be mounted on a computer. In the case of realization by a separate computer, only the inspection program is incorporated in the computer (inspection apparatus main body) 7, and the data of the spatial filter, which is automatically calculated by the computer for creating a spatial filter, is used for a computer (screening) for inspecting a screen defect. The inspection apparatus body 7 may be used.
[0057]
The present invention is not limited to a liquid crystal panel using the above-described TFT element, and displays such as a liquid crystal panel, a plasma display, an organic EL display, and a DMD (direct mirror device) using other diode elements. It can be used for inspection of body parts and display devices and products using them, and can be used not only for inspection of panels but also for inspection of printed matter such as printed matter. It goes without saying that it is not excluded from the scope of the invention.
[Brief description of the drawings]
FIG. 1 is a flowchart showing a processing flow of a genetic algorithm.
FIG. 2 is a flowchart showing a subroutine of an evaluation value calculation process in FIG. 1;
FIG. 3 is a flowchart showing a subroutine of a crossover process in FIG. 1;
FIG. 4 is a flowchart showing a subroutine of a mutation process in FIG. 1;
FIG. 5 is an image diagram of a solution expressed in the form of a gene.
FIG. 6 is an explanatory diagram of crossover processing.
FIG. 7 is an explanatory diagram of a mutation process.
FIG. 8 is an explanatory diagram in a case where two-dimensional filter data is developed in the form of a gene.
FIG. 9 is a schematic diagram of a pseudo defect image used for creating and verifying a GA.
FIG. 10 is a configuration diagram of a filter showing an example of data arrangement using symmetry.
FIG. 11 is an explanatory diagram showing a problem of a square filter.
FIG. 12 is a configuration diagram showing a data arrangement example of a circular filter.
FIG. 13 is a configuration diagram showing another example of data arrangement of a circular filter.
FIG. 14 is an explanatory diagram showing a possible range of filter data.
FIG. 15 is a diagram showing an example of a real number circular filter.
FIG. 16 is a diagram illustrating an example of an integer circular filter as a comparative example.
FIG. 17 is a diagram showing an example of a filter in which the solution of the genetic algorithm does not converge.
FIG. 18 is a diagram showing an example of a real number rectangular filter.
FIG. 19 is a configuration diagram of a screen defect inspection device according to an embodiment of the present invention.
FIG. 20 is a flowchart showing a stain defect detection process.
FIG. 21 is a schematic diagram of each image from an input image to a stain defect detection image.
[Explanation of symbols]
1 projector, 2 liquid crystal panels, 3 screens, 4 images, 5 pattern generators, 6 CCD cameras, 7 computers, 8 display devices, 10 inspection screens, 11 input images, 12 display area images, 13 original images (display area correction images) ), 14 background image, 15 inspection image (background difference image), 16 detection image, 20 pseudo-stain, 21 defect area, 22 normal area, 25 spot defect, 31 screen edge

Claims (9)

A spatial filter for image processing created by using a genetic algorithm, wherein a filter numerical value constituting said spatial filter is a real number. The spatial filter according to claim 1, wherein the filter numerical values are arranged symmetrically with respect to a filter center. 3. The spatial filter according to claim 1, wherein the filter numerical values are arranged substantially circularly with respect to a filter center. 4. The spatial filter according to claim 3, wherein data put in an area substantially equidistant from the center of the filter has the same numerical value. In a method of creating a spatial filter for image processing based on a genetic algorithm,
In an input image including a detection target, a detection region including a defective portion to be detected by a spatial filter, and a step of setting a plurality of partitioned detection regions of a region including a normal portion that may not be detected,
Encoding the data of the spatial filter and expanding it into one-dimensional data, thereby setting the genetic algorithm as a gene;
A step of filling the numerical value in the gene by generating a real random number,
Preparing a large number of said genes in combination of real numbers,
Performing a crossover process and / or a mutation process on the prepared gene to repeat generation alternation while generating a new gene;
Returning a new gene generated by the crossover process and / or the mutation process to the spatial filter by a procedure reverse to the encoding, and performing a filter process on the image of the partitioned detection region;
A gene pattern is obtained such that the difference between the calculation result of the detection area of the defective part in the filtering process and the absolute value of the calculation result of the detection area of the normal part becomes maximum and the calculation result in the flat part of the image becomes zero. Process and
A method for creating a spatial filter, comprising: In an input image including a detection target, a detection region including a defective portion to be detected by a spatial filter, and a function of setting a plurality of divided detection regions of a region including a normal portion that may not be detected,
By encoding the data of the spatial filter and developing it into one-dimensional data, a function of setting as a gene of a genetic algorithm,
A function to fill the numerical value in the gene by generating a real random number,
A function of storing a large number of the genes in a combination of real numbers and performing a crossover process and / or a mutation process on the stored genes to repeat generation alternation while generating a new gene. When,
A function of returning a new gene generated by the crossover process and / or the mutation process to the spatial filter by a procedure reverse to the encoding, and performing a filter process on the image of the partitioned detection region;
A gene pattern is obtained such that the difference between the calculation result of the detection area of the defective part in the filtering process and the absolute value of the calculation result of the detection area of the normal part becomes maximum and the calculation result in the flat part of the image becomes zero. A spatial filter creation program characterized by causing a computer to realize functions and functions. In the filter processing of image processing for detecting a screen defect to be inspected, the spatial filter according to any one of claims 1 to 4, the spatial filter created by the creating method according to claim 5, or the spatial filter according to claim 6. A screen defect inspection method using a spatial filter created by the creation program described above. Imaging a screen to be inspected;
Performing a filtering process for defect enhancement on the image captured by the imaging,
A step of setting a threshold value for the image after the filter processing and extracting defect candidates,
Has,
The spatial filter according to any one of claims 1 to 4, or the spatial filter created by the creating method according to claim 5, or the spatial filter created by the creating program according to claim 6. A method for inspecting a screen defect, characterized by using: An image pickup means for picking up an image of a screen to be inspected, and a computer incorporating the spatial filter creation program according to claim 7, wherein the computer or a computer other than the computer is a computer according to claim 7 or 8. A screen defect inspection apparatus configured to perform a defect inspection method.

Images