WO2008077680A1 - Method and device for optically testing objects - Google Patents

Abstract

The invention relates to a method and a device for testing an object with regard to deviations with respect to at least one reference object. Digital images of the object and of the reference object are subdivided into a multiplicity of image areas. For the image areas, according to a test criterion in each case at least one reference and test value are determined, which are compared with one another in order to determine deviations.

Description

 Method and device for the optical inspection of objects

The invention relates to a method and a corresponding apparatus for the optical inspection of objects.

The production of large quantities of consumer goods usually takes place by means of conveyor belts. On these, the consumer goods are manufactured or packaged, to then bring them on the market. In the filling and packaging systems, for example, a large number of identical containers or products are filled at high speed, sealed, labeled and then packed or collected in further packaging.

In order to reduce defective articles, they are subjected to a check during, but usually at least at the end of, production in order to eject defective articles from the further production process or to prevent them from being sold.

Control of all articles produced by a person who, for example, looks at all articles and visually inspects for errors, is ruled out at high production speeds. At higher production speeds, on the one hand, a person would have only a fraction of a second to look at an article, on the other hand, the person would quickly tire due to the monotonous activity.

By contrast, machine systems can be adapted to the high production and processing speeds with new technologies. For visual inspection image recognition and image processing methods are used. The article to be tested is guided past a camera which produces an image of the article, which is then processed further. By means of a CCD camera images can be generated, which can be transmitted in a simple manner to a data processing system. The data processing system then checks the image, for example, for deviations against a reference image. If the determined deviations exceed a predefined limit value, this can Data processing system trigger a predefined action, for example, to remove the recognized as faulty article from production.

In such machine recognition systems, different methods of digital image processing are used to process the images. For example, pattern or color recognition methods are used to verify the correct placement or color of features on the articles. Such methods are implemented in a software program which is run in the data processing system for each generated image.

For example, WO 01-09820 describes a method for digital image processing in which rectangular areas within an image are identified in which predefined operations are to be performed. These predefined operations are, for example, resizing to a predefined size, a color or gamma correction. Then, for each identified area, it is checked whether predefined features exist in it, for example features that can be interpreted as the presence or absence of features. In this test, the features determined for the image to be examined are compared with a reference in order to determine deviations or similarities.

WO2005 / 043141 A1 describes a system for testing material, for example networks, and in particular for optimizing the test method. In this test method, the nets are passed by cameras and generated images from both sides of the network, which are processed in a data processing system. From the generated images, for example, individual characteristic values are calculated, for example the gray value, which is compared with a previously determined reference gray value. The reference values are calculated as the average of a large number of individual values, each of which was determined on the basis of possibly faulty test pieces. A deviation of a value of an individual item is interpreted as an error if a statistically determined limit value is exceeded. Apart from the gray value mentioned, other characteristic features can be determined and tested. This method is optimized to reduce deviations erroneously recognized as errors by a person reviewing the evaluated reference images and comparing them with the detected errors. Pictures on which one Error is then removed and it will be determined from the sequence of error-free images new reference values. Thresholds whose overshoot or undershoot is interpreted as an error can still be changed manually.

A disadvantage of the methods described above is the associated computational effort, which, with limited computational resources, essentially determines the time required for examining an image and thus an article. The invention therefore describes a method and a corresponding device for fast and reliable testing of articles.

The invention describes a method for checking an object for deviations from at least one reference object, with the following method steps:

A. generating at least one reference digital image from the at least one reference object and a corresponding digital image from the object;

B. determining at least one statistical reference value according to at least one test criterion based on the reference image and a corresponding test value for the image;

C. comparing the at least one determined reference value with the at least one corresponding test value for determining deviations between the image and the at least one reference image, and

D. Evaluation of the deviations found, characterized in that

E. at least part of the reference image and a corresponding, equal part of the image are divided into several image areas, and that for each image area of the reference image at least one reference value and for each corresponding image area of the image, a corresponding test value is determined.

In the following, the method and the corresponding apparatus for carrying out the method will be described in more detail with reference to figures. Show it

Figure 1 is a schematic plan view of a device for carrying out the

process Figure 2 is a schematic representation of an object to be tested

Figure 3 is a schematic representation of the object image in a first

step

Figure 4 is a schematic representation of a division of an image in the first

image areas

Figure 5 is a schematic representation of a division of an image into first and second image areas

FIG. 1 shows a schematic plan view of a device 110 for carrying out the method. The objects 120 to be tested are moved on a conveyor belt 130 past a first and a second camera 140, 141 in the direction of the arrow. The basic body of the objects is aligned so that it is moved past the cameras in a defined orientation. In the illustrated example, the objects have an oval base body whose short axis 121 points to the cameras. The orientation serves to align the object, so that it can be ensured that a particular side of the object is viewed with the following procedure. The alignment can take place via any feature, for example, based on the outer shape or in an object round base, for example via a notch in the ground, as it is known from bottles.

The cameras each generate an image in front of a defined background 150, 151 from a moving object, wherein the first camera 140 generates an image from one side and the second camera 141 generates an image from the opposite side of the object. The generation of an image is triggered by means of a suitable triggering device 160, 161, when the object is moved past this. Such a triggering device may for example be a light barrier or a mechanical switch. When generating the images, for uniform and defined illumination of the object, flash devices 170, 171, preferably with LED lights, can be used. The cameras, such as CCD cameras, generate digital images, each composed of a finite set of pixels. A pixel, a so-called pixel, can reproduce the color of the imaged object in color or black and white, whereby in the case of a black-and-white reproduction, the color values are reproduced as grayscale values. The pixels are arranged in rows and columns so that an image is a matrix of pixels. In the device described here, CCD cameras with a resolution of 1392x1040 color pixels were used, so that the total image produced by the camera has 1392x1040 = 1,447,680 pixels. Basically, the use of cameras with different resolutions is possible, with a higher resolution allows the reproduction of smaller details of the imaged object or the use of a lower resolution with fewer pixels restricts the reproduction of the object to coarse details. As higher resolution cameras and data processing equipment will be more powerful in the future, it will be possible to use higher resolution cameras. The inventive method is not limited in terms of a higher resolution.

From each object to be tested so that images are generated, which are transmitted from the cameras in the form of image files to a data processing system 180. The transmission of the image files is preferably initiated by a camera, so that immediately after the generation of an image file, this is transmitted to the connected data processing system, in order then to be processed as quickly as possible. To transmit the image files, the cameras can be connected via a known interface to the data processing system, for example by means of a firewire IEEE 1394 interface. The image file can be in any format, for example, to reduce the amount of data to be transmitted in a packed format such. B. "jpg" or in an unpacked format such. B. "bmp" or in a proprietary format, which may be, for example, a specific format of the camera manufacturer, transmitted. The format conversion into a format suitable for processing the data then takes place in the data processing system. The data processing system 180 may be a so-called personal computer PC, in particular a laptop. The cameras as well as the device for ejecting an object identified as defective can be connected via the standard interfaces 190 normally present on the data processing system.

FIG. 2 shows an image 200, shown schematically here, of an object 210 against a background 220, as it is transmitted from a camera to the data processing system. The dashed line 201 in the drawing represents the outer dimensions of the image. The image shown here schematically as a line drawing is actually transmitted as a color image to the data processing system, so that all colors are reproduced.

As an alternative to a colored picture, the picture can also be a black-and-white picture, whereby the colors are rendered as grayscale.

The background 220 can have at least one vertical contrast jump, so that different objects stand out clearly at least from one of the contrasts and thus the outer contour can be determined. For this purpose, the background of the exemplary embodiment described here has, in its upper and lower regions, a light area and a dark area positioned therebetween.

In the example shown here, the object 210 is a - bottle-shaped vessel shown schematically - which is shown in frontal view. The object has at its upper end a closure 230 and on its front side a label 240. The vessel and the label may be colored in a commercial manner; this is not shown in the drawing.

When checking the objects, different values determined during the check of an individual object are compared with reference values. These reference values must therefore be present in the implementation of the method for each type of object to be checked so that they can be determined before the actual checking of objects. The determination of the reference values takes place on the basis of sample objects which have no errors or deviations or only permissible deviations. A number of these sample objects are processed by means of the test method, wherein the sample objects are not checked for deviations, but serve to determine reference values.

The process steps described below are carried out both when determining the reference values and when checking the objects.

FIG. 3 illustrates a first method step of processing the image 300 of a pattern object. In this first method step, the central axis 310 of the object 320 or the image of the object is determined. For this purpose, the first vertical contrast jump is first determined in each case within a surface 330 or 331 from outside to inside, ie from the image edge in the direction of the image center. For this purpose, a method known per se in image processing can be used which, for example, compares the color values of horizontally adjacent pixels. The surfaces 330, 331 are placed in such a way that they capture both a piece of the light background and the dark background, so that for a light color object, the contrast jump with respect to the dark background and, for an object of dark color, the contrast jump with respect to the light Background can be determined.

The determined contrast jumps are interpreted in each case as the outer edge of the object 320. The central axis 310 of the image of the object can now be determined mathematically simply and is formed by those pixels which lie in the middle between the determined outer edges.

As shown in FIG. 4, a grid 410 is subsequently placed on the image 400 of the object, wherein the grid is larger than the object image and the grid shape is coarsely adapted to the shape of the object image. The grid can be aligned with the previously determined central axis, so that the imaginary grid is always positioned the same.

Analogous to the determination of a vertical central axis here, a horizontal center axis can also be determined, which can be used in accordance with the vertical orientation of the imaginary grating. The fields of the grid are shaped so that they cover a surface without gaps, without overlapping. Accordingly, the grid fields have a triangular, hexagonal or preferably a quadrangular shape. The shape of the grid is chosen so that the image of the object is completely covered by the grid.

The grid fields thus define image areas 420, 430. For the pixels of each image area, statistical values are subsequently determined, which are stored as reference values for features of an image area. Those image areas - designated 420 by way of example - which at least partially represent the background are excluded from further processing. This rules out that parts of the background would be checked and evaluated as erroneous, even though the object itself is error-free. Accordingly, in the subsequent processing steps, only those image areas-designated by way of example by reference numeral 430-are processed and checked, which image exclusively the object.

The size of the image areas is useful to choose, taking into account various factors. Thus, the number of image areas must be small enough that their management in the data processing system does not take a disproportionate amount of time. On the other hand, the image areas must not be too large, so that the exclusion of the image areas 420 does not leave too much of the object unconsidered. The number of pixels per image area is in turn to be chosen such that a statistical evaluation over the pixels of an image area is statistically meaningful or significant. Furthermore, the resolution of the camera (in pixels), ie the number of pixels of the entire image has to be considered. For the exemplary embodiment described here, a number of 5,000 to 8,000 image areas has proven successful with a camera resolution of 1392 × 1040 pixels and an object size of approximately 70 × 320 mm.

With

Z = number of pixels in an image area;

N = number of pixels of the total image, corresponds to pixel d. Camera;

H = size of the object in millimeters; and

, wobei dieser Wert eine Richtgröße darstellt, die in Abhängigkeit von der zur Verfügung stehenden Rechenleistung der Datenverarbeitungsanlage, der Kameraauflösung und der gewünschten Verarbeitungsgeschwindigkeit variiert werden kann.0.1 </ <10, preferably 0.6 </ <3.5; For example, the number of pixels per image area can be determined

This value represents a guide variable that can be varied depending on the available computing power of the data processing system, the camera resolution and the desired processing speed.

For the determination of the reference values, it is optionally possible to increase the number of pixels for an image area which arrive at the determination of a value, the number of image areas to be taken into consideration and their location remaining unchanged, in which case the pixel of the image area is determined adjacent image areas are taken into account. For example, the adjoining two rows or columns of pixels of the adjacent image areas can be used to determine the reference values for the image area 431, so that more pixels than the image area itself are used to determine the reference values. In practice it has been found that this procedure increases the security in the determination of the reference values, wherein the amount of pixels lying outside an image area is adjustable.

For each image area 430, a plurality of reference values can now be determined and stored, wherein for different objects different different reference values can first be determined, stored and used in the actual test.

A reference value may, for example, turn off the color values of the pixels in a color space. For this purpose, the standard deviation of the red-green-blue values of the pixels is determined, so that for each color results in a value for the average value and the scattering around this mean value.

Another reference value can be determined by means of the Sobel operator, which provides a measure of the roughness of an image area. The Sobel operator, which is known per se, determines differences in brightness between the pixels, which can be interpreted as edges. For each pixel, a value is calculated by means of the Sobel operator, which in turn is used to calculate an average value and the standard deviation. Furthermore, the red-green-blue values (RGB) supplied by the camera for calculating the color saturation, intensity and brightness can be transformed into other color spaces, for example the HSV or HLS color space, in order to obtain the mean value and the associated standard deviation of the color saturation and intensity and to determine the brightness.

In this way, for each image area for different features, e.g. the mean and standard deviation of the colors, the Sobel operator as well as the color saturation and intensity values are determined. In addition to the features listed here by way of example, various other per se known features of a matrix of contiguous pixels can be used as features for the test.

A certain number of - well found - sample objects are now processed to generate the reference values. For each image area of each pattern object, the determined mean values and standard signatures are stored, so that a number of determined mean values and associated standard deviations are available for each image area and each feature corresponding to the number of pattern objects.

If, for example, k different features M are to be taken into account when checking an object, and corresponding reference values are determined on the basis of a number / sample objects, the mean value and the associated standard deviation are determined for each feature M for each feature.

M ₁ = [M ₁₁ , ..., ^]

M _k = [M _kl , ..., M _kl ]

After the determined reference values and standard deviations have been determined, the values in the intervals M _γ ,..., M _{k are} sorted in size. From these values determined for each image area and each feature, the median, ie the mean value of an interval, is then determined as the reference value for the respective feature M. The Selection of the respective median ensures that extreme values have no influence on the reference values.

The selected reference values are then stored permanently, so that they only have to be loaded when restarting the system or the data processing system in order to be able to check associated objects immediately. Also, reference values for various types or types of objects may be stored so that the device for testing corresponding objects may be set up very quickly by loading the corresponding reference values.

After the reference values for the individual features to be checked have been determined for the image areas, the inspection of the objects can begin. For this purpose, as in the determination of the reference values for each view of an object to be examined, an image is generated by the cameras and transferred to the data processing system.

In the data processing system, the same method steps are used on the transferred images or image files as for determining the reference values, wherein the values now determined for the image areas do not serve as reference values, but the test values are the test values.

Accordingly, in a first step, the center axis of the respective imaged object is determined by the method described above. At this a - thought - grid is aligned, which divides the object image into image areas. For each of the image areas, the values are then determined for the same features or, if configured accordingly in the software of the data processing system, only for some of the features, again taking into account those image areas that map at least part of the background.

For each feature value of an image area is then checked whether this is within a predetermined range of values. The limits of the range of values are initially predetermined by the standard deviation of the reference value, but can be changed manually, so that larger or smaller deviations from the respective reference value can be set as tolerable. If the test shows that the value of the feature is within the permissible limits, this is considered "good". Otherwise, if the feature value determined for an image area is outside the permissible limits, this is rated as "erroneous" and an error counter is incremented. This error counter is, for example, always incremented by 1 if at least one feature value is outside the predetermined interval for an image area, but at most once for an image area, so that in the case of multiple deviations within an image area the error counter is only incremented once. The error counter thus indicates the number of image areas in which at least one value of a feature is outside the permissible interval.

Whether a tested object is to be assessed as defective and accordingly ejected is determined by means of the error counter. If an image has too many image regions with deviations, that is to say the error counter has a correspondingly high counter reading, the object under test belonging to the image is taken out of further production or ejected by means of a suitable device which is connected to the data processing system ,

In order to test an image section of the object image, it can be divided into second image areas. In practice, it has been shown that objects to be tested are, for example, covered with labels. The labels are produced in a separate production process and then glued by a roll using a suitable machine. In this case, the position of the label on the object in the horizontal and / or vertical direction vary and thus differ from that on the reference objects. If, for example, a label has a color contrast to the color of the background and deviates the position of a label, for example in the horizontal direction, such that the vertical edge of the label falls into a different column of first image areas, for example, they would be erroneously identified as being defective be rated.

In order to test such a section, in this case of the label, within the image of the object, it is thus advantageously possible to lay a second - imaginary - grating over the image section so that its grating fields intersect the section in the second Divide image areas, which are tested separately with the method steps described above.

As shown in Figure 5, a second grid 510 is placed over the label 540 so that the second image areas defined by the second grid 510 completely cover the label. The second image areas may have the same size as the first image areas or be larger or smaller. The size of the image areas can also be chosen such that the image detail whose geometric dimensions are known in the image in many cases, is split exactly into second image areas, so that the edges of the second grid on the edge of the image section or just within the edges lies.

For exact placement of the second grid, for example, a center line of the detail can be determined and used or one or two reference points or reference numbers 550 can be used, which are printed on each label at the same location. In this embodiment, this is the silhouette of the head 550. In order to align the grid, the exact placement of the silhouette is now determined within an area in which the silhouette is usually placed using a conventional algorithm in image processing. Since the placement of the reference character relative to the label is known, now the second - imaginary - grid can be placed exactly.

If the outer dimensions of the second grid project beyond the outer dimensions of the label, ie if the second grid is selected larger than the label, then second image areas overlap at least partially with first image areas. These second image areas are not taken into account in the subsequent processing, but are removed from the processing and thus from the test. This prevents parts of the bottle body from being checked in the case of an improperly placed label. Since these second image areas each map changing sections of the bottle body due to the inaccurate placement, an examination of these second image areas would erroneously lead to a high number of errors.

Analogously, first image regions which completely or at least partially overlap with second image regions do not become defective during the processing of the first image regions considered. This prevents correspondingly that in the examination of these first image areas due to the inaccurate placement of the label, the first image areas from object to object represent very different sections of the label and would falsely lead to a high number of errors.

As for the first, a group of image areas can be determined manually for the second image areas by means of the software program running on the data processing system, which are excluded from the examination. Thus, for example, the group of those second image areas which at least partially represent the lettering 560 can be excluded from the test.

The examination of the second image areas takes place analogously to the examination of the first image areas. Accordingly, for the pixels of each image area, a reference value is initially determined for at least one feature to be checked on the basis of pattern objects and the method described above. Subsequently, for each of the image areas to be processed, the value for each feature to be tested is determined and compared with the reference feature value. If a comparison between one for an image area and the corresponding feature reference value results in a deviation beyond the permissible range, a corresponding counter is increased.

The counter may be the one that was increased during the examination of the first image areas, or it may be another, separate counter out. In the case of separate, independent counters, corresponding limit values for the maximum number of errors allowed for an object and / or area can be defined and evaluated independently of each other before the object belonging to a region to be tested is rejected as defective from further production.

In an analogous manner, a plurality of superimposed or juxtaposed grids can be used to define corresponding further image areas which are tested in the manner described above.

The method can also be designed such that individual image regions from the first or further image regions are excluded from the test. This can for example be done by an image of a pattern object with associated Division into image areas on a monitor of the data processing system is displayed and manually a group of image areas or individual image areas can be marked so that for this no check is performed. In this way, for example, those areas of an image can be taken out of the test, in which a variable lettering, such as a date stamp, is attached. Such variable lettering would in many cases, for example when using the Sobel filter, always lead to different values that would be erroneously considered as errors.

While the test described above assumes images taken with visible light waves, it is also possible to process infrared images or images that image UV light.

The described method thus enables fast, reliable testing of objects for visible deviations of sample objects, with which the reference values for the respective features were previously determined. The automated generation of reference values based on sample objects offers a flexible adaptation to different objects, so that the method and the corresponding device can be adapted quickly and without great effort to the examination of various objects. Likewise, the storage of once obtained and proven in practice reference values allows the maintenance of consistent product quality, the sensitivity of the method by manually changing the interval limits for a reference value, the fast and flexible adaptation to a desired product quality.

Claims

claims 1. A method for checking an object 120 for deviations from at least one reference object, comprising the following method steps: A. generating at least one reference digital image 400 from the at least one reference object and a corresponding digital image 400 from the object 120; B. determining at least one statistical reference value according to a test criterion based on the reference image and a corresponding test value for the image 400; C. comparing the at least one determined reference value with the at least one corresponding test value for determining deviations between the image and the at least one reference image, and D. Evaluation of the determined deviations, characterized in that E. at least part of the reference image and a corresponding, same part of the image in several image areas 420, 430 are divided, and that for each image area 420, 430 of the reference image at least one reference value and for each corresponding image area of the image 400, a corresponding test value is determined , 2. The method of claim 1, wherein for dividing the part of the reference image and the corresponding part of the image 400, these are divided into adjacent image areas 420, 430. 3. The method according to any one of the preceding claims, wherein the size of the image areas 420, 430 is determined in pixels

With Z = number of pixels in an image area; N = number of pixels of the overall picture; H = size of the object in millimeters; and 0.1 </ <10, preferably 0.6 </ <3.5; 4. The method of claim 1, wherein the reference object and the object have at least one reference point 550 and the image areas are placed relative to the reference point 550. 5. The method according to any one of the preceding claims, wherein a vertical or horizontal central axis 310 of the image of the reference object or corresponding to the object is determined to place the portion of the reference image to be split in image areas 420, 431 and corresponding to the image. 6. The method of claim 5, wherein for generating the reference image, the reference object and corresponding to the generation of the image 400, the object in front of a background 150, 151 are shown, which has at least one horizontal and / or vertical contrast jump. 7. The method of claim 6, wherein for determining the horizontal / vertical center axis, the horizontal / vertical outer edges of the reference object are determined in an image area 330, 331, the horizontal / vertical outer edges of the reference object or according to the object and at least one vertical / horizontal Contrast jump of the background 150, 151 has. 8. The method according to any one of the preceding claims, wherein for determining the at least one reference value, a respective value for a plurality of reference objects is determined and is selected from the plurality of values of the median as a reference value. 9. The method according to any one of the preceding claims, wherein it is checked in step C, whether the test value is within a predefined value interval around the reference value. 10. The method of claim 9, wherein the limits of the value interval are manually changeable. 11. The method according to any one of the preceding claims, wherein for determining the number of different image areas 430, a counter is incremented. 12. The method according to any one of the preceding claims, wherein image areas that at least partially reflect the background 150, 151 are not checked. 13. The method according to any one of the preceding claims, wherein for checking a portion of the object image, this is divided into second image areas 510 and these are checked in accordance with the examination of the first image areas. 14. The method of claim 1, wherein those second image areas 510 that overlap with first image areas 420, 430 or image the background 150, 151 are not checked. 15. The method according to any one of the preceding claims, wherein those first image areas 430 which at least partially overlap with first image areas are not checked. 16. Method according to one of the preceding claims, wherein the reference values for the image regions of the reference object and the test values for the corresponding image regions 430 of the image 400 are respectively determined as an average over all pixels of the respective image region. 17. The method according to any one of the preceding claims, wherein in the determination of a reference value for an image area additionally pixels of adjacent image areas are taken into account. 18. The method of claim 16 or 17, wherein the test criterion is a base color or the brightness or the color saturation in a color space or the value of the Sobel operator. 19. The method according to any one of the preceding claims, wherein the determined reference values stored on a disk and from a disk can be loaded. 20. Device for checking an object for deviations from at least one reference object with at least one camera for generating digital images of the object and the reference object and a data processing system connected to the camera for processing the images, wherein the data processing system by means of a software program for processing the images according to one of the preceding claims is set up.