JP2008292430A - Appearance inspection method and appearance inspection apparatus - Google Patents

Abstract

In an appearance inspection method and an appearance inspection apparatus, a simple configuration eliminates the need to input data for comparison, reduces the amount of calculation processing, shortens processing time, high-speed processing, supports complicated appearance shapes, and high Enables accurate visual inspection.
An appearance inspection apparatus 1 includes a camera 3 that captures a two-dimensional image G, a projector 4 for measuring three-dimensional data, and a calculation processing unit 20. A defect candidate portion detection unit 11 includes a two-dimensional image. A defect candidate part is extracted from G, the first three-dimensional data generation means 12 generates three-dimensional data P1 which is the three-dimensional shape data of the inspected object 10, and the second three-dimensional data generation means 13 The three-dimensional data P2 of the area excluding the defect candidate portion is generated from the three-dimensional data P1, and the third three-dimensional data generation means 14 generates the three-dimensional data P3 obtained by interpolating the value of the defect candidate portion in the three-dimensional data P2. Then, the defect determination means 15 determines a defect in the defect candidate portion based on the difference value δZ between the three-dimensional data P1 and P3.
[Selection] Figure 1

Description

  The present invention relates to a method for inspecting the appearance of a workpiece mainly in a production line, and an appearance inspection apparatus for performing the method.

  Conventionally, in a production line such as a resin molded product, automatic inspection by image processing is performed instead of visual appearance inspection. In the appearance inspection, foreign matter, discoloration, scratches, etc. on the product surface are detected, thereby repairing the product and preventing the outflow of defective products. Molded products with dust on the surface are considered good products by repairing by wiping off the dust in the subsequent process, and molded products with foreign matter buried in the resin or discolored resin cannot be repaired. The

  Therefore, in the automation of appearance inspection, it is necessary to accurately identify dust on the product surface, foreign matter embedded in the product surface, discoloration of the product surface, and the like. When the accuracy of identification is poor, the product yield can be reduced by considering a repairable product as defective and discarded, or the number of work steps can be increased by the operator re-inspecting visually. These are all useless costs and lead to an increase in product costs.

  By the way, in the appearance inspection using a two-dimensional image that has been conventionally performed, for example, in the case of a plain product, the inspection can be performed with relatively high accuracy using luminance information. However, as described above, there is a limit to the processing of a two-dimensional image when it is desired to further increase inspection accuracy and reduce unnecessary costs. Therefore, for example, measurement of a three-dimensional surface shape of a product is considered effective for detecting dust attached to the surface.

As an example of performing an appearance inspection using three-dimensional measurement data, there is an appearance inspection of a soldered state in a process of solder mounting a surface-mounted component on a printed circuit board. For example, an appearance inspection method for collating a master pattern based on three-dimensional shape data of a component to be soldered with three-dimensional measurement data measured by a three-dimensional image sensor after component mounting (for example, see Patent Document 1), cream An external appearance in which an inspection frame surrounding a pad for printing solder is set and stored in advance, and a solder bridge is detected based on whether or not the solder extension measured by three-dimensional measurement after component mounting intersects the inspection frame. An inspection device is known (see, for example, Patent Document 2).
JP-A-7-91932 JP 2004-317291 A

  However, even if the inspection method as shown in Patent Document 1 described above can be applied to an appearance inspection related to a simple-shaped component mounted at a specific place, any inspection method such as adhesion of foreign matters in the appearance inspection of a molded product can be used. There is a problem when trying to apply it to the inspection of defects occurring in a place. That is, in such an inspection method, the shape data stored for the three-dimensional shape of the entire part to be inspected and the measurement data need to be aligned and collated in the three-dimensional space. When the three-dimensional shape becomes complicated, the amount of data representing the three-dimensional shape increases. Compared with a three-dimensional space takes a lot of processing time compared to a two-dimensional case, so an increase in the amount of data is a heavy burden. In addition, since the size of the defect is usually smaller than the size of the molded product, the background amount (bias component) increases when the entire product is verified, making it difficult to produce a difference in the matching rate indicating the level of verification. It is difficult to detect.

  Further, in the inspection apparatus as shown in Patent Document 2, since the location of the defect of the solder bridge generated from the solder pad can be specified, the inspection position can be set in advance around a specific inspection frame. However, in the case of inspection of a defect that occurs in an arbitrary place such as adhesion of foreign matter, it is necessary to set an inspection frame in all the places where the defect may occur, and it takes time to set the inspection frame. There is a problem that it takes a long time for inspection because it is necessary to inspect even a portion where no occurrence occurs.

  The present invention solves the above-described problems, eliminates the need for input of comparison data with a simple configuration, reduces the amount of calculation processing, shortens the processing time, supports high-speed processing, and complex appearance shapes. It is an object of the present invention to provide an appearance inspection method and an appearance inspection apparatus capable of realizing a highly accurate appearance inspection.

  In order to achieve the above object, the invention of claim 1 is an appearance inspection method for inspecting the presence or absence of defects in the appearance of an object to be inspected, the step of imaging the object to be inspected and generating a two-dimensional image; Detecting a region different from a predetermined luminance value in the two-dimensional image as a defect candidate portion; and three-dimensional shape data of the inspection object viewed from the same viewpoint as the viewpoint from which the two-dimensional image was captured, Acquiring first three-dimensional data composed of data corresponding to each pixel of the two-dimensional image, setting a measurement line passing through the defect candidate portion and its periphery in the two-dimensional image, and the first three Extracting data corresponding to the measurement line in a region excluding the defect candidate portion from the dimensional data to generate second three-dimensional data; and the second three-dimensional data A step of obtaining third three-dimensional data composed of three-dimensional data obtained by interpolating three-dimensional data of a region corresponding to the defect candidate portion, and a difference between the first three-dimensional data and the third three-dimensional data Determining a value for data corresponding to the measurement line, and determining the presence or absence of a defect in the defect candidate portion based on the magnitude of the difference value.

  According to a second aspect of the present invention, in the appearance inspection method according to the first aspect, the second three-dimensional data is a predetermined range from an end of the defect candidate portion in the first three-dimensional data. Thus, it is generated by extracting from data corresponding to a range not including a point where the curvature of the surface of the object to be inspected is discontinuous.

  According to a third aspect of the present invention, in the visual inspection method according to the first or second aspect, the step of determining the presence or absence of the defect adds each difference value obtained for the data corresponding to each point of the measurement line. Thus, the step of obtaining the total difference sum and the step of comparing the total difference sum with a preset threshold value are included.

  According to a fourth aspect of the present invention, in the visual inspection method according to the first or second aspect, in the step of determining the presence or absence of the defect, each difference value obtained for the data corresponding to each point of the measurement line is added. Calculating the total difference sum, determining the number of pixels of the defect candidate portion for which the total difference sum is to be calculated, dividing the total difference sum by the number of pixels to determine an average difference value, and the average A step of comparing the difference value with a preset threshold value.

  The invention of claim 5 is an appearance inspection apparatus using the appearance inspection method according to any one of claims 1 to 4.

  According to the first aspect of the present invention, the defect presence / absence is determined by comparing the three-dimensional data limited to the defect candidate portion detected by the image processing of the two-dimensional image, so that the inspection can be performed at high speed with a small amount of calculation processing. Can do. In addition, since inspection is performed based on three-dimensional data, discrimination between foreign matter attached to the surface and foreign matter embedded in the surface or a discolored portion of the surface can be realized with high accuracy based on detection of unevenness. . These cannot be identified by conventional two-dimensional image processing. The determination of the presence / absence of a defect based on the difference between the first and third three-dimensional data is performed by comparing the maximum value of the difference in the defect candidate portion with a predetermined threshold, or by comparing the sum of the differences in the defect candidate portion with a predetermined value. This can be easily done by comparison with a threshold value.

  Further, since the data (third 3D data) obtained by interpolating the data of the defect candidate part from the data of the peripheral part is used as the comparison target data of the 3D shape, that is, the 3D data of the good product shape, Therefore, it is not necessary to input the three-dimensional data as comparison data and to secure a memory capacity for the storage. Furthermore, when the third three-dimensional data is obtained by interpolation, processing is performed along a preset measurement line, so that data can be easily obtained in a short processing time. As described above, since the appearance inspection can be easily performed with less data and processing time, it is possible to easily perform the high-precision appearance inspection even for the inspected object having a complicated appearance shape which requires more data and processing time. . As the measurement line, a plurality of lines passing through the defect candidate portion and its periphery can be set.

  According to the second aspect of the present invention, the second three-dimensional data is obtained by limiting to a region that is within a certain range from the edge of the defect candidate portion and does not include a point where the curvature of the surface shape is discontinuous. There is little data and processing is easy. In addition, since the second three-dimensional data does not include a discontinuity of curvature and the curved surface obtained as the third three-dimensional data is a smooth curved surface, the non-defective shape three-dimensional data (third three-dimensional data) Is easily and accurately obtained by interpolation. Therefore, it is possible to realize a highly accurate appearance inspection.

  According to the invention of claim 3, since the inspection of the defect candidate portion is performed using the total difference sum, the inspection can be stably performed without being affected by the change in the size or shape of the defect portion.

  According to the invention of claim 4, since the inspection of the defect candidate portion is performed using the average difference value, it is possible to perform the stable inspection without being affected by the variation of the defect shape.

  According to the invention of claim 5, with a simple configuration, the calculation processing amount can be reduced, the processing time can be shortened, the high-speed processing, and the complicated appearance can be realized without performing the input operation of the comparison data. Accurate appearance inspection can be performed.

  Hereinafter, an appearance inspection apparatus and an appearance inspection method according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 shows a block configuration of an appearance inspection apparatus, and FIGS. 2A to 2C show examples of a two-dimensional image and three-dimensional data in an appearance inspection method performed using the appearance inspection apparatus.

  The appearance inspection apparatus 1 is an apparatus that efficiently performs the presence or absence of defects in the appearance of the inspection object 10 using the two-dimensional image G and the three types of three-dimensional data P1, P2, and P3. The foreign matter adhering to the surface and the defects such as surface scratches and discoloration are distinguished and detected.

  The appearance inspection apparatus 1 includes a camera 3 that captures a two-dimensional image G, a projector 4 that emits illumination light having an intensity of a sine stripe pattern, and an appearance inspection apparatus body 2 that performs calculation processing and control. .

  The appearance inspection apparatus main body 2 operates the calculation processing unit 20 that performs data processing to determine the presence or absence of a defect, the image generation unit 21 that processes image data from the camera 3 to obtain a predetermined image, and the projector 4. A sine fringe pattern generation unit 22 that generates a sine fringe pattern, an input / output unit 23 that receives an instruction and data input from a user and outputs data to an external device, an image of a defective portion on an external display 24a, A display creation unit 24 that generates data for displaying inspection results and the like, and a storage unit 25 that stores image data, inspection conditions, inspection results, and the like are provided.

  The calculation processing unit 20 includes defect candidate portion detection means 11, first, second, and third three-dimensional data generation means 12, 13, and 14, and defect determination means 15, and performs data processing. The determination result of the presence or absence of defects is output to the input / output unit 23 and the display 24a. Hereinafter, each means of the calculation processing unit 20 will be described by paying attention to processing.

  The defect candidate part detection means 11 extracts a defect candidate part from the two-dimensional image G. The two-dimensional image G is picked up by the camera 3 and includes pixel luminance distribution data on the xy coordinate plane as shown in FIG. The defect candidate part detection means 11 detects an area different from the predetermined luminance value in the two-dimensional image G as a defect candidate part.

  The first three-dimensional data generation means 12 processes the three-dimensional data acquired by the sine fringe pattern generation unit 22, the projector 4, the camera 3, and the image generation unit 21, and generates first three-dimensional data P1 (hereinafter referred to as “first three-dimensional data P1”). , Simply referred to as three-dimensional data P1, and so on). Details of the generation of the three-dimensional data P1 will be described later (see FIGS. 10, 11, and 12).

  The three-dimensional data P1 generated as described above is three-dimensional shape data of the inspected object 10 viewed from the same viewpoint as the viewpoint from which the two-dimensional image G is captured, as shown in FIG. It consists of data corresponding to each pixel of the two-dimensional image G. That is, the three-dimensional data P1 is data indicating the distribution of the Z coordinate values instead of the luminance distribution in the two-dimensional image G.

  The second three-dimensional data generation means 13 generates the three-dimensional data P2 shown in FIG. 2C from the three-dimensional data P1. The three-dimensional data P2 is data generated by extracting corresponding data in a region excluding the above-described defect candidate portion from the three-dimensional data P1. That is, the data of the area m corresponding to the above-described defect candidate portion in the three-dimensional data P2 is set to a predetermined value, for example, a zero value.

  As shown in FIG. 2D, the third three-dimensional data generation means 14 interpolates the value of the area m in the above-described three-dimensional data P2 by interpolating based on the values around the area m. The three-dimensional data P3 is generated.

  The defect determination means 15 obtains a difference value δZ related to the Z value of the two three-dimensional data P1 and P3 described above, and determines the presence / absence of a defect in the defect candidate portion based on a magnitude comparison between the difference value δZ and a predetermined threshold value. judge. That is, when the difference value δZ is larger than the threshold value, it is determined that the foreign substance is attached to the defect candidate portion, and it is concluded that there is no defect. This is because the foreign matter adhering to the surface of the object to be inspected 10 can be removed by a process such as wiping or blowing off, and thus is different from a non-repairable defect such as discoloration or scratch.

  Such an appearance device 1 is data obtained by interpolating the data of the defect candidate portion from the data of the peripheral part as the comparison target data of the three-dimensional shape, that is, the three-dimensional data of the good product shape, that is, the third three-dimensional data P3. Therefore, it is not necessary to input three-dimensional data of a good product shape as comparison data in advance and to secure a memory capacity for storing the same.

  Further, as the three-dimensional data P2 and P3, data generation and data comparison can be performed by limiting to the defect candidate portion and its peripheral region. In this case, it is only necessary to process the three-dimensional data in a limited area, and data processing can be easily performed in a short processing time. Accordingly, since the appearance inspection can be easily performed with a small amount of data and the processing time, a highly accurate appearance inspection can be easily performed even for an inspected object having a complicated appearance shape that requires more data and processing time.

  In addition, the interpolation of the three-dimensional data in the region m shown in FIG. 2C can use an approximate plane such as an approximate plane or a quadric surface. In this case, an estimation method using the least square method can be used.

  2A to 2D, plane processing is performed as the three-dimensional data P2 and P3. However, one or a plurality of measurement lines passing through the defect candidate portion are defined, and the measurement lines are related to the measurement lines. Three-dimensional data P2 and P3 may be generated, and the presence or absence of a defect may be determined by comparing the three-dimensional data P1 and P3 with respect to the measurement line.

  As described above, when the measurement line is used, when the three-dimensional data P3 is obtained by interpolation, the process may be performed along the measurement line set in advance, and the defect determination may be performed along the measurement line. Data can be obtained more easily in a short processing time, and appearance inspection can be easily performed with less data and processing time. Therefore, a more accurate appearance inspection can be easily performed even on an object having a complicated appearance shape that requires more data and processing time.

  Next, an appearance inspection method using the above-described measurement line will be described in the order of steps with reference to FIGS. 3 to 6 together with FIG. 3A, 3B, and 3C show an outline of processing of a two-dimensional image. FIGS. 4A and 4B show more specific first three-dimensional data and a two-dimensional image. 5 shows the first three-dimensional data in the cross section including the measurement line, and FIGS. 6 (a1) to (a4) and (b1) to (b4) are the first, second, and third in the cross section including the measurement line. 3D data and difference value data are shown.

(Two-dimensional image G)
A two-dimensional image G shown in FIG. 3A is an image taken using the camera 3 of the appearance inspection apparatus 1 shown in FIG. The camera 3 images the object to be inspected 10 from above, and the image data is converted into a two-dimensional image G by the image generation unit 21, and the luminance value (gray value) of each pixel is stored as a pixel value based on the xy coordinates. 25. Here, the luminance value I of each region in the two-dimensional image G is such that I1 = 100 in the region M1, I2 = 170 in the region M2, and Ib = 180 in the background region other than the regions M1 and M2. To do.

(Detection of defect candidate part)
In the two-dimensional image G described above, a region different from the predetermined luminance value is detected as a defect candidate portion. Here, the range of the predetermined luminance value is 160 <predetermined luminance value <200.

  The defect candidate part detection means 11 compares the luminance value in each pixel of the two-dimensional image G with the above-described predetermined luminance value range, and detects a pixel having a luminance value outside the above-mentioned range as a defect candidate part. Then, since the luminance value I1 = 100 in the region M1 is outside the predetermined range, the region A1 is detected as a defect candidate portion (hereinafter, defect candidate portion A1) as shown in FIG. When the defect candidate portion is not detected, the inspected object 10 is determined to be a non-defective product, and the following appearance inspection process is completed. When a defect candidate part is detected, the first three-dimensional data P1 is acquired.

(First three-dimensional data P1)
First three-dimensional data P1 is obtained which is the three-dimensional shape data of the inspected object 10 viewed from the same viewpoint as the viewpoint at which the above-described two-dimensional image G is captured, and which includes data corresponding to each pixel of the two-dimensional image G. Is done. The same camera 3 that captured the two-dimensional image G can be used for acquiring the three-dimensional data P1 by using a phase shift method described later. As a result of using the same camera 3, the pixels of the two-dimensional image G, the pixels of the three-dimensional data P1, and the points on the inspection object 10 have a one-to-one correspondence. Three-dimensional data of the surface shape of the object to be inspected 10 is obtained by a procedure described later.

  When these two-dimensional image G and three-dimensional data P1 are displayed on the liquid crystal display, the coordinates (u, v) on the liquid crystal screen, the coordinates (x, y) on the image, and the coordinates (X, Y) in the three-dimensional space. , Z) is not originally limited, but for the sake of simplicity, the relationship between the coordinates on the liquid crystal screen and the coordinates on the image is the x-axis on the image and the u axis on the liquid crystal screen. Assume that the axes are parallel. Further, it is assumed that the x-axis and y-axis on the image are parallel to the X-axis and Y-axis in the three-dimensional space, respectively, and the Z-axis is in the optical axis direction of the camera.

(Measurement line setting)
When the above-described defect candidate part A1 is detected, as shown in FIG. 3C, in the two-dimensional image G, a plurality of defect candidate parts A1 and a plurality (six in this example) passing through the periphery are parallel. Measurement lines L1 to L6 are set. The measurement line is set by the calculation processing unit 20 shown in FIG. These measurement lines are set in parallel to the x-axis direction for the sake of simplicity of processing under the assumption of the above-described coordinate system, but are not limited to this, and may be in other directions or arbitrary curves. it can.

  Incidentally, the inspected object 10 shown in FIG. 1 and FIGS. 3A to 3C has a substantially flat surface, and is intended for defects and foreign matters on the surface. In the following, the appearance inspection method will be described more generally using the example of the inspected object shown in FIG. The three-dimensional data P1 shown in FIG. 4 (a) includes a concave slope 10a having a smooth curved surface, an upper flat face 10b, and a ridge line 10c composed of intersecting lines on both sides, and foreign matter M adheres to the concave slope 10a. is doing. In the two-dimensional image G shown in FIG. 4B, the defect candidate portion A corresponding to the foreign matter M and six measurement lines L1 to L6 passing therearound are set as described above. In addition, six measurement lines L1 to L6 corresponding to the two-dimensional image are also shown in the three-dimensional data P1 shown in FIG.

(Second three-dimensional data P2)
As shown in FIGS. 4A and 4B, when the measurement line is set and the three-dimensional data P1 is acquired, the measurement line L1 in the region excluding the defect candidate portion A from the three-dimensional data P1 and the like. The data corresponding to is extracted to generate second three-dimensional data P2. The second three-dimensional data generation unit 13 generates the three-dimensional data P2.

  In order to efficiently generate the above-described three-dimensional data P2 and the subsequent processing, the three-dimensional data P2 is generated with the data amount suppressed as much as possible. This will be described with reference to FIG. FIG. 5 shows a change in height in the Z-axis direction with respect to the X-axis in the cross section including the measurement line L3 in FIG. In this figure, it can be seen that the curvature of the smooth curve corresponding to the concave slope 10a changes discontinuously at the position corresponding to the ridge line 10c. Moreover, the convex part corresponding to the foreign material M is shown on the smooth curve corresponding to the recessed part slope 10a.

  In the first place, the second three-dimensional data P2 is based on the assumption that the defect candidate portion A is a result of the foreign matter M adhering to the surface of the inspected object. This gives basic data for estimating the surface of the material. Therefore, the three-dimensional data P2 may be generated in the range up to the peripheral area of the defect candidate portion A. Furthermore, it is preferable not to include data of a region where the curvature changes discontinuously, such as the ridge line 10c shown in FIG.

  Therefore, in the situation shown in FIGS. 4A and 4B, as shown in FIG. 5, the three-dimensional data P2 in the region W including the regions dx1 and dx2 on both sides of the region w of the foreign matter M along the measurement line. It is preferable to produce it. In this case, the region W is a region that does not include the region of the ridge line 10c. That is, the three-dimensional data P2 corresponds to a predetermined range within the three-dimensional data P1 from the end of the defect candidate portion A and does not include a point where the curvature of the surface of the inspection object is discontinuous. Generated by extracting from the data.

  Since the three-dimensional data P1 and P2 extracted along the measurement line L1 in FIG. 4A described above have no defect candidate portion on the measurement line L1, as shown in FIGS. 6A1 and 6A2, It is a gentle curve indicating the surface shape of the object to be inspected (concave slope 10a).

  Further, since the three-dimensional data P1 extracted along the measurement line L3 has the defect candidate portion A on the measurement line L3, as shown in FIG. 6B1, a gentle curve of the surface shape of the object to be inspected. Is a curve having a convex portion reflecting a defect shape having a width w. In addition, as shown in FIG. 6B2, the three-dimensional data P2 extracted along the measurement line L3 has the width w corresponding to the defect candidate portion on the measurement line L3 as the surface shape of the inspection object. It is a curve that falls out of a gentle curve.

  Here, the procedure for generating the second three-dimensional data P2 will be described with reference to FIG. 5 again. When the second three-dimensional data generating means 13 reads (that is, extracts) the three-dimensional data P2 from the three-dimensional data P1, the three-dimensional corresponding to the center of gravity (xg, yg) in the two-dimensional image G of the defect candidate portion A Starting from the X coordinate value Xg in space, data is first read in the positive direction of the X axis. Each time the data is read, the second three-dimensional data generation means 13 calculates the curvature at that point, for example, the difference between the previous and next Z values, and whether the reading reaches a predetermined reading range (the right end of the region W) Data reading is terminated when it becomes larger than the determined threshold value and becomes discontinuous. Next, data reading and curvature calculation are similarly performed from the coordinate value Xg in the negative direction of the X axis, and the reading reaches a predetermined reading range (the left end of the region W) or the curvature is larger than a predetermined threshold value. When it changes and becomes discontinuous, the data reading is terminated. Such reading is performed for each measurement line.

  By using the reading method as described above, it is possible to read data of a portion where the amount of data to be read is smaller and the height data (Z value) changes smoothly as the second three-dimensional data P2.

(Third three-dimensional data P3)
When the second three-dimensional data P2 is generated, the third three-dimensional data generating unit 14 includes the three-dimensional data formed by interpolating the three-dimensional data of the region corresponding to the defect candidate portion A in the three-dimensional data P2. 3 three-dimensional data P3 is generated. The three-dimensional data P3 represents the three-dimensional shape of the inspection object in a state where the defect candidate part A does not exist. That is, the three-dimensional data P3 includes a collection of curves and straight lines approximating the three-dimensional data P2 generated for each measurement line in a plane including the measurement lines. Examples of the three-dimensional data P3 are shown in FIGS. 6 (a3) and (b3).

(Defect determination)
When the first three-dimensional data P1 and the third three-dimensional data P3 are generated, the defect determination means 15 makes the Z of both the three-dimensional data P1 and P3 as shown in FIGS. 6 (a4) and (b4). A difference value δZ is calculated for each measurement line. The defect determination means 15 determines what kind of defect the defect candidate portion is based on the magnitude of the obtained difference value δZ.

  The simplest determination method is to compare a predetermined threshold value with the difference value δZ, and when there is a difference value δZ larger than the threshold value, the defect candidate portion A has a foreign object attached to the surface of the object to be inspected. It is the method of determining that it is. In addition, the maximum value of the difference value δZ in the defect candidate portion A can be obtained and compared with a threshold value, or the sum or average value of the difference values δZ can be compared with the threshold value. Details of these will be described later (see FIGS. 7, 8, and 9).

  According to the appearance inspection method using the two-dimensional image G and the three-dimensional data P1, P2, and P3 by setting the measurement lines as described above, the processing is performed along the measurement lines set in advance, so that less data and processing time are required. This makes it possible to easily inspect the appearance. A plurality of measurement lines can be set as a line passing through the defect candidate portion and its periphery, limited to the necessary minimum area. Therefore, the defect candidate portion A based on the two-dimensional image G and the processing target area are limited to the peripheral area, the target is further limited on the measurement line, and the comparison of the three-dimensional data P1 and P3 is performed to determine whether there is a defect. Therefore, the inspection can be performed at a high speed with a small amount of calculation processing. In addition, it is possible to distinguish between foreign matter adhering to the surface and foreign matter embedded in the surface and discolored portions on the surface with high accuracy based on the detection of irregularities, which could not be identified only by two-dimensional image processing. .

  Further, the data obtained by interpolating the data of the defect candidate portion A from the data P2 of the peripheral portion of the defect candidate portion A as the comparison target data of the three-dimensional shape, that is, the non-defective shape three-dimensional data P3, that is, the third three-dimensional data Since the data P3 is used, it becomes unnecessary to input the three-dimensional data of a good product shape as comparison data in advance and to secure a memory capacity for storing the data.

  Further, since the second three-dimensional data P2 is obtained only in a region that is within a certain range from the edge of the defect candidate portion A and does not include a point where the curvature of the surface shape is discontinuous, the third three-dimensional data The shape of the curved surface obtained as P3 is expected to be a smooth curved surface, and the shape can be easily and accurately obtained by interpolation. Therefore, it is possible to realize a highly accurate appearance inspection.

  In the present invention, first, a defect candidate portion A that is likely to have “foreign matter adhesion” or “foreign matter contamination” is detected using the two-dimensional image G, and then the three-dimensional data P1 around the defect candidate portion A is used. Then, when there is no defect in the defect candidate portion A, that is, the three-dimensional data P3 indicating the three-dimensional shape in the case of a non-defective product is estimated, and then the good product three-dimensional data P3 estimated in the defect candidate portion A and From the difference from the measured three-dimensional data P1, if something is placed on the three-dimensional shape of the non-defective product, it is determined as “attachment of foreign matter”, and if the same as the three-dimensional shape of the good product, It is determined that foreign matter is mixed. Thus, since the state of the foreign matter is identified using not only the luminance value on the two-dimensional image G but also three-dimensional data, it is possible to perform highly reliable identification.

  By the way, taking the appearance inspection in the production process of the conventional resin molded product as an example, it is as follows. Each product molded by an injection molding machine is inspected for appearance of “chips”, “scratches”, “cracks”, “foreign matter contamination”, and “foreign matter adhesion” before shipment. In this appearance inspection, when a defect presence / absence inspection is performed by a two-dimensional image processing apparatus based on a luminance value, the detection threshold for defect detection seems to be overdetected so as to make a determination on the safe side so that defective products are not put on the market. Set to Therefore, since an operator has to re-inspect a product that is over-detected as a defective product even though it is a non-defective product, it is a problem to reduce the number of products to be re-inspected. The normal production process is not a clean room, and dust adheres when the product passes through the process line. In the visual inspection of the worker, it is easy to identify whether it is dust that can be removed by wiping or foreign matter mixed in the resin, but it is a two-dimensional detection that uses a difference in luminance value to detect defects. The image processing apparatus cannot distinguish between these, and both are determined as the same foreign substance defect.

  Therefore, by using the appearance inspection method of this embodiment, the distinction between “foreign matter adhesion” and “foreign matter contamination” can be easily realized, and the operator confirms only those judged as “foreign matter adhesion” and removes the foreign matter. Since it is not possible to remove the foreign matter “mixed with foreign matter” by wiping it off and making it a non-defective product, it is possible to dispose it as a defective without bothering the operator's hand, and it is possible to reduce both the work man-hours and cost.

  Next, an example of the process of the appearance inspection method will be described with reference to the flowchart of FIG. As described above, after obtaining the two-dimensional image G (S1) and detecting the defect candidate portion from the two-dimensional image G (S2), the presence / absence of the defect candidate portion is determined (S3), and the defect candidate is detected. When there is no part (No in S3), it is determined as a non-defective product (S4), and the appearance inspection is completed.

  When a defect candidate part is detected (Yes in S3), the first three-dimensional data P1 is acquired (S5), and a plurality of measurement lines passing through the defect candidate part and its periphery are set on the two-dimensional image G. (S6).

  Next, for each measurement line, data that does not include data corresponding to the defect candidate portion is extracted from the first three-dimensional data P1, and second three-dimensional data P2 is generated (S7).

  Next, for each measurement line, third three-dimensional data P3 that is data obtained by interpolating the data of the defect candidate portion in the second three-dimensional data P2 is generated (S8).

  Next, with respect to each measurement line, a difference value δZ between the first three-dimensional data P1 and the third three-dimensional data P3 is calculated, and each difference value δZ is added on each measurement line. As a result, the total difference sum V is calculated (S9).

Here, the calculation of the total difference sum V will be described. The measurement lines are identified by k, and the total number is m (k = 1 to m), the number of data included in each measurement line k is n (k), and the data identification variables on each measurement line are j = 1 to n. (K), the Z value of the first three-dimensional data P1 is Z _P1 (k, j), and the Z value of the third three-dimensional data P3 is Z _P3 (k, j). V is expressed as in equation (1). Note that δZ = Z _P1 (k, j) −Z _P2 (k, j).

  The total difference sum V obtained by the above equation (1) is compared with a predetermined threshold value α (S10). If the total difference sum V is larger than the threshold value α (α <V, Yes in S10), the foreign matter is attached. It is determined (S11). This determination of foreign matter adhesion is a determination that the object to be inspected is a non-defective product, that is, a determination that there is no defect in the defect candidate portion.

  On the other hand, if the total difference sum V is equal to or less than the threshold value α (V ≦ α, No in S10), it is determined that there is no foreign matter adhering, and therefore, it is determined that there is a defect, for example, foreign matter contamination (S12). This determination is a determination that the defect candidate portion has a defect.

  Next, another example of the process of the appearance inspection method will be described with reference to the flowchart of FIG. The processes from steps S1 to S9 in this example are the same as the processes from steps S1 to S9 shown in the flowchart of FIG. In this example, defect determination is performed using the average difference value H obtained by dividing the above-described total difference sum V by the total number of pixels S of the target defect candidate portion. Therefore, regarding the measurement line, the pixel number S of the defect candidate portion is calculated (S21), and the average difference value H is calculated by dividing the total difference sum V by the total pixel number S (S22). The average difference value H is a so-called pseudo height.

  Next, the average difference value H is compared with a predetermined threshold value β (S23). If the average difference value H is larger than the threshold value β (β <H, Yes in S23), it is determined that foreign matter is attached (S24). This determination of foreign matter adhesion is a determination that the object to be inspected is a non-defective product, that is, a determination that there is no defect in the defect candidate portion.

  On the other hand, if the average difference value H is equal to or less than the threshold value β (H ≦ β, No in S23), it is determined that no foreign matter is attached, and therefore, it is determined that there is a defect, for example, foreign matter mixing (S25). This determination is a determination that the defect candidate portion has a defect.

  Next, still another example in the process of the appearance inspection method will be described with reference to the flowchart of FIG. The processes from steps S1 to S22 of this example are the same as the processes of steps S1 to S22 shown in the flowchart of FIG. 8, and are other examples using a pseudo height.

  In this example, two-stage determination is performed on the average difference value H using two threshold values β0 and β1. That is, in the first stage determination, if the average difference value H is larger than a predetermined threshold β1 (β1 <H, Yes in S30), it is determined that foreign matter is attached (S31). This determination of foreign matter adhesion is a determination that the object to be inspected is a non-defective product, that is, a determination that there is no defect in the defect candidate portion.

  If the average difference value H is not greater than the predetermined threshold value β1 (No in S30), the second stage determination is performed. That is, if the average difference value H is in a range between predetermined threshold values β0 and β1 (where β0 <β1) (β0 <H ≦ β1, Yes in S32), it is determined that foreign matter is mixed (S33). . This determination of contamination is a determination that the object to be inspected is defective, that is, a determination that the defect candidate portion has a defect.

  If the average difference value H is not in the range between the predetermined threshold values β0 and β1 (No in S32), it is determined that the defect candidate portion is a scratch or a chip (S34). In other words, when the average difference value H is smaller than or equal to the predetermined threshold value β0 (H ≦ β0), it is determined as a scratch or a chip. The determination of the scratch or the chip is a determination that the object to be inspected is defective, that is, a determination that the defect candidate portion has a defect.

(Phase shift method)
Next, a phase shift method, which is a method for obtaining the above-described three-dimensional data P1, will be described with reference to FIGS. FIGS. 10A to 10D show changes in the intensity of an image captured by projecting a sine fringe pattern used when acquiring three-dimensional data, and FIGS. 11A and 11B are obtained by sine fringe pattern illumination light. Shows lighting. Please refer to FIG. 1 again.

  In the phase shift method, imaging performed by irradiating an object to be inspected with illumination light having a sine fringe pattern is performed a plurality of times while shifting the phase of the sine fringe pattern, and the luminance value of the fringe pattern in the obtained multiple images is changed. Is a three-dimensional measurement method for obtaining a three-dimensional shape of an object to be inspected based on the above. By this phase shift method, data (Z coordinate value) of the first three-dimensional data P1 is obtained for each pixel of the two-dimensional image G.

  Hereinafter, a four-phase phase shift method for changing the phase of the sine fringe pattern by ¼ period will be described. Note that the phase shift method includes a method using three phases in which the phase is changed by 1/3 period, and a method using five phases or more. Any of these methods can be used to acquire the three-dimensional data P1.

The sine fringe pattern data described above is generated in the sine fringe pattern generation unit 22 in the appearance inspection apparatus 1 shown in FIG. Is projected from the projector 4 onto the inspected object 10. The illumination light has a fringe pattern whose brightness changes in a sine wave shape. The sine fringe pattern data L _i (u, v) generated by the sine fringe pattern generation unit 22 is expressed as in Expression (2).

Here, the coordinates (u, v) are coordinates on the liquid crystal surface of the projector 4. The sine fringe pattern data L _i (u, v) has a bias b (u), an amplitude a (u), a phase φ (u), and a phase shift amount iπ / 2 (I = 0, 1, 2, 3). It is represented. It is sufficient to change the brightness of illumination light having a sine stripe pattern only in the u direction. Therefore, L _i (u, v) does not include the v component in the bias, amplitude, etc., and has the same value in the v direction.

In acquiring the three-dimensional data P1, first, the camera 3 captures an image of the inspection object 10 irradiated with illumination light having a sine fringe pattern in a certain phase state, that is, the inspection object 10 onto which the sine fringe pattern is projected. The imaging data is converted into a two-dimensional plane image by the image generation unit 21, and the luminance value I _i (x, y) of each pixel is recorded in the storage unit 25. Next, the phase state of the sine wave is changed by π / 2, and the sine fringe pattern is projected and imaged, and the luminance value is recorded as described above. Finally, the luminance values I _i (x, y), (i = 0, 1, 2, 3) of a total of four two-dimensional images obtained by changing the value of i in the above formula (2) to 0-3. ) Is taken into the storage unit 25.

  FIGS. 10A to 10D show changes in the luminance value I in the x-axis direction in the four two-dimensional images captured in the storage unit 25 described above. Further, an example of a two-dimensional image captured by projecting a sine fringe pattern onto a plane is shown in FIG. 11A, and a two-dimensional image captured by projecting a sine fringe pattern onto a plane having a circular convex portion on the surface is shown. An example is shown in FIG.

Next, the phase φ (x, y) of the fringe pattern in each pixel is the luminance value I _i (x, y), (i = 0, 1, 2, 3) of each pixel of the four two-dimensional images described above. ) Is calculated by the calculation processing unit 20 as the following expression (3). The coordinates (x, y) are coordinates on the image.

Here, I _i (x, y), (i = 0, 1, 2, 3) is the same as in equation (2), and bias B (x), amplitude A (x), phase φ (x), phase It is described by the following equation (4) using the shift amount iπ / 2 (i = 0, 1, 2, 3).

  The sinusoidal fringe pattern projected onto the object to be inspected is modulated (distorted) according to the three-dimensional surface shape of the object to be inspected, for example, as shown in FIG. However, there is a correspondence between the phase φ (u) at the time of projection and the phase φ (x, y) obtained from the captured image. Therefore, the correspondence between the coordinates (x, y) on the image and the coordinates (u, v) on the liquid crystal is obtained, and the first three-dimensional data P1 on the surface of the object to be inspected, that is, (X, Y) by the principle of triangulation. , Z) value is obtained from the following equation (5).

Here, (X, Y, Z) are coordinates in the three-dimensional space that is the three-dimensional data P1, u (φ) is a function that associates the phase with a pixel on the liquid crystal surface, and P ^c is obtained by camera calibration. optical information is a parameter having the posture information in a three-dimensional space, P ^P is the optical information obtained by the calibration of the projector, a parameter having a posture information in a three-dimensional space, these following formula ( 6) and Expression (7) is satisfied.

  An example in which this technique is actually applied is shown in FIGS. This is an example in which foreign matter is adhered on the curved surface of the object to be inspected. 12A shows an example of the first three-dimensional data P1, FIG. 12B shows an example of the third three-dimensional data P3, and FIG. 12C shows the first three-dimensional data P1. The difference value δZ of the third three-dimensional data P3 is shown. As described above, even if there is a defect (attached foreign matter) on the curved surface, only the foreign matter portion can be detected and the defect can be identified.

  The present invention is not limited to the above-described configuration, and various modifications can be made. For example, as the method for acquiring the first three-dimensional data P1, the phase shift method has been described above. However, the three-dimensional shape measurement method can acquire the three-dimensional data P1 corresponding to the same pixel as the two-dimensional image G. For example, it can be used instead of the phase shift method. For example, the first three-dimensional data P1 may be generated from scan data using laser light. Further, the difference value δZ, and thus the total difference sum V and the average difference value H, may be negative if, for example, a recess due to a scratch exists in the defect candidate portion.

  Further, the order of processing can be interchanged between the step of generating a two-dimensional image, the step of detecting a defect candidate portion, and the step of acquiring the first three-dimensional data. In addition, since the measurement line is set to minimize the processing data, the measurement line only needs to conform to this point. That is, it is possible to make a measurement line with pixels thinned out, or to set a measurement line in parallel at a position several pixels away. In addition to the method based on the difference in luminance, a method based on color information such as saturation and chromaticity can be used as the defect candidate portion detection method.

The block block diagram of the external appearance inspection apparatus which concerns on one Embodiment of this invention. (A) is a figure which shows the example of a two-dimensional image in the appearance inspection method which concerns on one Embodiment of this invention using an external appearance inspection apparatus same as the above, (b) is a figure which shows the example of the 1st three-dimensional data, (c). The figure which shows the 2nd three-dimensional data example, (d) is a figure which shows the 3rd three-dimensional data example. (A) is a figure which shows the other example of a 2D image same as the above, (b) is a figure which shows the example which extracts a defect candidate part from the 2D image, (c) is a mode that a measurement line is set to a 2D image FIG. (A) is a figure which shows the other example of 1st three-dimensional data, (b) is a figure of the two-dimensional image corresponding to the same three-dimensional data. The graph which shows the 1st three-dimensional data in the cross section containing the measurement line L3 of Fig.4 (a). (A1) is a graph showing the first three-dimensional data in the cross section including the measurement line L1 in FIG. 4 (a), (a2) is a graph showing the second three-dimensional data, and (a3) is the third A graph showing three-dimensional data, (a4) is a graph showing a difference value between the first and second three-dimensional data, and (b1) is a first three-dimensional in a cross section including the measurement line L3 in FIG. (B2) is a graph showing the second three-dimensional data, (b3) is a graph showing the third three-dimensional data, and (b4) is a graph showing the first and second three-dimensional data. The graph which shows a difference value. The flowchart of the process in an external appearance inspection method same as the above. The flowchart which shows the other example of the process in an external appearance inspection method same as the above. The flowchart which shows the further another example of the process in an external appearance inspection method same as the above. (A)-(d) is a figure which shows the intensity | strength change of the image imaged by projecting a sine fringe pattern, when acquiring the 1st three-dimensional data in an external appearance inspection method same as the above. (A) is a figure which shows the mode of illumination of the flat surface by the same sine fringe pattern illumination light, (b) is a figure which shows the mode of illumination of the flat surface with a convex part partially by the same sine fringe illumination light. (A) is a three-dimensional diagram showing an example of the first three-dimensional data, (b) is a three-dimensional diagram showing an example of the third three-dimensional data, and (c) is a diagram of the first and third three-dimensional data. The three-dimensional figure which shows a difference value.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Appearance inspection apparatus 10 Inspected object A, A1 Defect candidate part G 2D image H Average difference value L, L1-L6 Measurement line M Defect P1 1st 3D data P2 2nd 3D data P3 3rd 3 Dimensional data S Number of pixels V Total difference sum α, β, β1, β2 Threshold value δZ Difference value

Claims (5)

An appearance inspection method for inspecting the appearance of an inspection object for defects.
Imaging the object to be inspected to generate a two-dimensional image;
Detecting a region different from a predetermined luminance value in the two-dimensional image as a defect candidate portion;
Obtaining first three-dimensional data consisting of data corresponding to each pixel of the two-dimensional image, which is three-dimensional shape data of the inspected object viewed from the same viewpoint as the viewpoint from which the two-dimensional image was captured; ,
Setting a measurement line passing through the defect candidate portion and its periphery in the two-dimensional image;
Extracting the data corresponding to the measurement line in the region excluding the defect candidate portion from the first three-dimensional data to generate second three-dimensional data;
Obtaining third three-dimensional data comprising three-dimensional data obtained by interpolating three-dimensional data of a region corresponding to the defect candidate portion in the second three-dimensional data;
Obtaining a difference value between the first three-dimensional data and the third three-dimensional data for data corresponding to the measurement line, and determining the presence or absence of a defect in the defect candidate portion based on the magnitude of the difference value. And an appearance inspection method characterized by comprising:   The second three-dimensional data is a predetermined range from the end of the defect candidate portion in the first three-dimensional data, and does not include a point at which the curvature of the surface of the inspection object is discontinuous. The appearance inspection method according to claim 1, wherein the appearance inspection method is generated by extracting from data corresponding to a range. The step of determining the presence or absence of the defect includes adding each difference value obtained for data corresponding to each point of the measurement line to obtain a total difference sum;
The visual inspection method according to claim 1, further comprising a step of comparing the total difference sum with a preset threshold value. The step of determining the presence or absence of the defect includes adding each difference value obtained for data corresponding to each point of the measurement line to obtain a total difference sum;
Obtaining the average difference value by finding the number of pixels of the defect candidate portion as a target for obtaining the total difference sum and dividing the total difference sum by the number of pixels;
The visual inspection method according to claim 1, further comprising a step of comparing the average difference value with a preset threshold value.   An appearance inspection apparatus using the appearance inspection method according to claim 1.

Images