CN1314049A - Method and apparatus for inspection of printed circuit boards - Google Patents

Abstract

An image acquisition system including a plurality of sensors each including a multiplicity of sensor elements, pre-scan calibration subsystem, employing a predetermined test pattern (108), which is sensed by the plurality of sensors (100), for providing an output indication in two dimensions of distortions in the output of the plurality of sensors, the output indication being employed to generate a function which maps the locations viewed by the sensor elements in at least two dimensions and a distortion subsystem operative during scanning of an article by the plurality of sensors to correct the distortions by employing the output indication.

Description

Printed circuit board inspection method and equipment

Field of the invention

The present invention generally relates to inspection systems and methods for articles, and more particularly relates to inspection systems and methods for general two-dimensional articles such as printed circuit boards.

Background of the invention

Various article inspection systems and methods are known in the patent and professional literature. A well-known problem with existing inspection systems is that great difficulty is encountered when trying to strictly arrange multiple line image sensors to jointly acquire an image of a single line of the inspected item. Other issues include: pixel elongation due to camera angles, difficulty combining overlapping images from multiple cameras, and factors such as physical separation of diodes in a line sensor, chromatic aberration, and different wavelengths of light. The shift of each color component of the image caused by factors such as the different intensity of the image.

The following patent documents are believed to represent the current state of the art:

下述专利号的各个美国专利：RE33,956,3,814,520,3,956,698,4,100,570,4,152,723,4,167,728,4,185,298,4,223,346,4,269,515,4,277,175,4,277,802,4,326,792,4,347,001,4,389,655,4,421,410,4,448,532,4,449,818,4,459,619,4,465,939,4,506,275 ,4,532,650,4,538,909,4,556,317,4,585,351,4,589,140,4,590,607,4,595,599,4,597,455,4,618,938,4,633,504,4,635,289,4,653,109,4,675,745,4,692,812,4,701,859,4,712,134,4,751,377,4,758,782,4,758,888,4,762,985,4,771,468,4,772,125,4,821,110,4,821,110,4,870,505 ,4,870,505,4,877,326,4,878,736,4,893,346,4,894,790,4,897,737,4,897,795,4,929,845,4,930,889,4,938,654,4,958,307,4,969,038,4,969,198,4,978,974,4,978,974,4,979,029,4,984,073,4,989,082,4,989,082,5,023,714,5,023,917,5,067,012,5,067,162,5,085,517,5,091,974 ,5,103,105,5,103,257,5,114,875,5,119,190,5,119,439,5,125,040,5,127,726,5,128,753,5,129,014,5,131,755,5,136,149,5,144,132,5,144,132,5,144,448,5,150,422,5,150,423,5,161,202,5,162,866,5,162,867,5,163,128,5,170,062,5,175,504,5,181,068,5,198,778,5,204,918 ,5,220,617,5,245,421,5,253,307,5,458,706,5,285,295,5,303,064,5,305,080,5,331,397,5,373,233,5,379,350,5,414,534,5,414,534,5,44,478,5,450,204,5,483,359,5,483,603,5,494,535,5,500,746,5,539,444；

The following European patent documents: EP094,501A2, EP598,582A2, EP426,182A2, EP426,166A2, EP247,308A2, EP243,939A2, EP206,713A2, EP128,107A1, EP126,492A2, EP426,166A2, EP533,348A2 ,EP209,422A2,EP536,918A2,EP92306649.2; and

British patent documents: GB2,201,804A and GB2,124,362A.

The following patent documents are believed to be most relevant to this application:

U.S. Patent Nos.: 4,459,619, 4,465,939, 4,675,745, 4,692,812, 4,821,110, 4,870,505, 5,144,132, 5,144,448, 5,438,359, and 5,500,746.

Summary of the invention

It is an object of the present invention to provide an improved article inspection system and method characterized by significantly increased accuracy.

Accordingly, a preferred embodiment of the present invention provides an image acquisition system comprising a plurality of sensors, a pre-scan calibration subsystem, and a distortion correction subsystem, wherein each of the plurality of sensors is Contains a plurality of sensor units; the pre-scan calibration subsystem provides an output indication of two-dimensional directional distortion in the output of the plurality of sensors by utilizing a predetermined test pattern detected by the plurality of sensors, the output indication being used to generate A function that maps the location "seen" by each sensor unit in at least two dimensions; the distortion correction subsystem can use the output indications to correct for distortion as the multiple sensors scan the item.

According to another preferred embodiment of the invention, the plurality of sensors comprises a plurality of sensors with different spectral sensitivities.

According to yet another preferred embodiment of the present invention, the plurality of sensors comprises at least two sensors having substantially the same spectral sensitivity.

According to yet another preferred embodiment of the present invention, the plurality of sensors includes at least two sensors that are at least partially overlapped in at least one direction.

According to yet another preferred embodiment of the present invention, the pre-scan calibration subsystem is capable of operating with sub-pixel precision.

According to yet another preferred embodiment of the present invention, the distortion correction subsystem performs non-zero order interpolation on the pixels in the outputs of the plurality of sensors.

In accordance with yet another preferred embodiment of the present invention, the distortion correction subsystem is capable of compensating for variations in pixel size across multiple sensors.

According to yet another preferred embodiment of the present invention, the distortion correction subsystem is capable of compensating for variations in magnification among multiple sensors.

According to yet another preferred embodiment of the present invention, the distortion correction subsystem is capable of compensating for chromatic aberration among multiple sensors.

According to yet another preferred embodiment of the present invention, the plurality of sensors comprises a plurality of sensors with different spectral sensitivities, and the above function depends on different integration times applied to the sensors with different spectral sensitivities.

According to yet another preferred embodiment of the present invention, the distortion correction subsystem is capable of compensating for variations in the shape of the pixels in the plurality of sensors.

According to yet another preferred embodiment of the present invention, the distortion correction subsystem is capable of operating with an accuracy better than 5% of the pixel size of the plurality of sensor units.

According to a preferred embodiment of the present invention, there is also provided an image acquisition system, the system includes: a plurality of sensors, a pre-scan calibration subsystem and a distortion correction system, wherein each sensor in the plurality of sensors contains multiple sensor units; the pre-scan calibration subsystem provides an output indicator of distortion in the output of multiple sensors by utilizing a predetermined test pattern detected by multiple sensors, and the output indication is used to generate a correction function; distortion correction The system can use the output indications to correct for distortion as multiple sensors scan an item, and works with an accuracy better than 5% of the pixel size of the multiple sensor units.

According to a preferred embodiment of the present invention, there is also provided an image acquisition system, the system includes: a plurality of sensors, a pre-scan calibration subsystem and a distortion correction subsystem, wherein the pre-scan calibration subsystem utilizes a A predetermined test pattern detected by the plurality of sensors moving relative to the plurality of sensors in a direction of relative motion to provide an output indication of distortion in the output of the plurality of sensors, and the pre-scan calibration subsystem is capable of The "seen" image of at least one target on the test pattern is correlated to determine the relative orientation of the plurality of sensors; the distortion correction subsystem can use this output indication to correct for distortion.

According to another preferred embodiment of the present invention, the pre-scan calibration subsystem is also capable of providing an output indicative of the orientation of the plurality of sensors relative to the scan direction.

According to yet another preferred embodiment of the present invention, each sensor of the plurality of sensors contains a plurality of sensor units, and the pre-scan calibration subsystem is also capable of determining pixel size characteristics of each sensor unit of each sensor.

In accordance with yet another preferred embodiment of the present invention, the pre-scan calibration subsystem is capable of determining each sensor's value by causing the sensors to "look" at a grating of parallel, equally spaced lines formed on the test pattern. The pixel size characteristics of a sensor unit.

According to a preferred embodiment of the present invention, there is also provided an image acquisition system, the system includes: a plurality of sensors, a pre-scan calibration subsystem and a distortion correction subsystem, wherein each sensor in the plurality of sensors contains multiple sensor units; the pre-scan calibration subsystem provides an output indication of two-dimensional distortions in the multiple sensor outputs by utilizing a predetermined test pattern detected by the multiple sensors, which is used to generate an energy map The position "seen" by the sensor unit in at least two directions; the distortion correction subsystem can use this output indication to correct for distortion when multiple sensors scan an item.

According to another embodiment of the present invention, the distortion correction subsystem can employ a user-selectable pixel size.

According to a preferred embodiment of the present invention, an article inspection system is also provided, which includes an image acquisition subsystem, an image analysis subsystem and an output indication subsystem, wherein the image acquisition subsystem is used to obtain an image of an inspected article image; an image analysis subsystem for identifying at least one predetermined characteristic of the item from the image; an output indication subsystem for providing an output indication of the presence of at least one predetermined characteristic of the article, characterized in that the camera assembly comprises: a plurality of A sensor assembly, a self-calibration device, and a sensor output correction device, wherein the self-calibration device is used to determine the geometric relationship between the various sensor components; the sensor output correction device is used to correct multiple sensors according to the geometric relationship between the various sensor components The output of the sensor assembly, and the sensor output correction device includes an electronic interpolation device capable of performing non-zero order interpolation on pixels in the output of the plurality of sensor assemblies.

According to a preferred embodiment of the present invention, there is also provided an item inspection system, the system includes: an image acquisition subsystem, an image analysis subsystem, and an output indication subsystem, wherein the image acquisition subsystem is used to acquire a An image of the inspected item; the image analysis subsystem is used to identify at least one predetermined characteristic of the article from the image; the output indication subsystem is used to provide an output indication of the presence of at least one predetermined characteristic of the article, wherein the camera assembly includes : a plurality of sensor components, self-calibration equipment and sensor output correction equipment, wherein the self-calibration equipment is used to determine the geometric relationship between the various sensor components; the sensor output correction equipment is used to correct the geometric relationship between the various sensor components Outputs of multiple sensor components, and the sensor output correction device can correct the outputs of multiple sensor components with sub-pixel precision.

A preferred embodiment of the present invention also provides an object inspection system, which includes: an image acquisition subsystem, an image analysis subsystem, and an output indication subsystem, wherein the image acquisition subsystem is used to acquire an inspected an image of the item; an image analysis subsystem for identifying at least one predetermined characteristic of the item from the image; an output indication system for providing an output indication of the presence of the at least one predetermined characteristic of the item, characterized in that the camera assembly comprises at least one A sensor assembly and a sensor output correction device, wherein the sensor output correction device is for modifying at least one output of the at least one sensor assembly based at least in part on an optical distortion associated with the at least one sensor assembly.

According to another preferred embodiment of the present invention, the optical distortion includes pixel size distortion.

According to yet another preferred embodiment of the present invention, the optical distortion comprises magnification distortion.

According to yet another preferred embodiment of the present invention, the optical distortion comprises chromatic aberration.

According to yet another preferred embodiment of the present invention, the optical distortion comprises an overlay mismatch.

According to yet another preferred embodiment of the present invention, the optical distortion includes pixel shift caused by sensor separation.

According to yet another preferred embodiment of the present invention, the optical distortion includes focus inconsistency for various color components.

According to yet another preferred embodiment of the present invention, the optical distortion comprises cumulative shifts for the various colors.

According to a preferred embodiment of the present invention, there is also provided an article inspection system, the system includes: an image acquisition subsystem, an image analysis subsystem and an output indication subsystem, wherein the image acquisition subsystem is used to acquire an inspected an image of an item; an image analysis subsystem for identifying at least one predetermined characteristic of the item from the image; an output indication system for providing an output indication of the presence of at least one predetermined characteristic of the item, characterized in that the camera assembly comprises at least A sensor assembly and sensor output correction device, wherein the sensor output correction device is adapted to correct at least one output of at least one sensor, and the sensor output correction device includes a sensor for generating a map capable of mapping a position on the sensor assembly to a set of scan positions A function generator for functions of .

According to a preferred embodiment of the present invention, an article inspection system is also provided, which includes: a camera assembly, an image analysis subsystem and an output indication subsystem, wherein the camera assembly is used to obtain an image of an inspected article; the image an analysis subsystem for identifying at least one predetermined characteristic of the item from the image; an output indication subsystem for providing an output indication of the presence of the at least one predetermined characteristic of the article, characterized in that the camera assembly comprises: a A user interface for the resolution of images acquired by the camera assembly, an electro-optic sensor assembly, and an electronic resolution modifier operating on the downstream side of the electro-optic sensor assembly.

According to another preferred embodiment of the present invention, the camera component can determine the pixel size of the image according to the resolution selection on the user interface.

A brief description of the drawings

Through the following detailed description combined with accompanying drawings, the present invention can be more fully understood and recognized, in the accompanying drawings:

Figure 1 is a simplified block diagram illustrating an article inspection system constructed and operative in accordance with a preferred embodiment of the present invention;

Fig. 2 is a simplified illustration of various parts of a preferred test pattern, some of which are not drawn to scale, which also shows a simplified representation of the field of view of each line sensor "viewing" the test pattern;

Figure 3 is a simplified block diagram illustrating a mapping function generator circuit which is a constituent part of the system of Figure 1;

Figure 4A is a simplified flowchart illustrating the operation of the pixel size and shape determination function of the mapping function generator circuit of Figure 3;

Figure 4B is a diagram illustrating the geometric distortion handled by the function of Figure 4A;

FIG. 4C is a simplified semi-graphical, semi-curvilinear illustration of the functionality illustrated in the flowchart of FIG. 4A;

5A is a simplified flowchart illustrating the operation of the test pattern angle determination function of the mapping function generator circuit of FIG. 3;

Figure 5B is a diagram illustrating the geometric distortion handled by the function of Figure 5A;

FIG. 5C is a simplified semi-graphical, semi-curvilinear illustration of the functionality illustrated in the flowchart of FIG. 5A;

6A is a simplified flowchart illustrating the "sensor relative orientation determination" operation performed by the "sensor field edge test pattern target image correlation" function of the mapping function generator circuit of FIG. 3;

Figure 6B is a diagram illustrating the geometric distortion handled by the function of Figure 6A;

Figure 6C is a semi-graphical, semi-curvilinear simplified illustration of the functionality illustrated in the flowchart of Figure 6A;

7A is a simplified flow diagram illustrating the "Sensor X Overlap and Y Offset Determination" operation performed by the "Adjacent Image Correlation" function of the mapping function generator circuit of FIG. 3;

Figure 7B is a diagram illustrating the geometric distortion handled by the function of Figure 7A;

Figure 7C is a diagram illustrating the functionality illustrated in the flowchart of Figure 7A;

Figure 8A is a simplified flowchart illustrating the operation of the "Multicolor X and Y Offset Determination" function of the mapping function generator circuit of Figure 3;

Figure 8B is a diagram illustrating the geometric distortion handled by the function of Figure 8A;

Figure 8C is a diagram illustrating the functionality illustrated in the flowchart of Figure 8A;

The combination of Figures 9A and 9B is a simplified flow diagram illustrating a preferred method of implementing the mapping function generator circuit of the system components of Figure 1;

FIG. 10A is a diagram illustrating acquisition of an object image by multiple cameras under ideal conditions;

Figure 10B is a diagram illustrating an image buffer for storing the image acquired in Figure 10A;

FIG. 11A is a diagram illustrating that fields of view of multiple cameras capturing an image of an object are mutually skewed and overlapped;

FIG. 11B is a diagram illustrating an image buffer for storing the image acquired in FIG. 11A;

FIG. 12 is a diagram to help understand the Y resampling function of the image correction circuit 110 in FIG. 1;

The combination of Figures 13 and 14 is a simplified diagram to help understand the overlap correction function of the image correction circuit 110 in Figure 1;

The combination of Figures 15 and 16 is a simplified diagram to help understand the X resampling function of the image correction circuit 110 in Figure 1; and

FIG. 17 is a diagram to help understand the characteristics of the cumulative offset correction function of the image correction circuit 110 in FIG. 1 .

Detailed Description of the Preferred Embodiment

Referring now to FIG. 1, there is shown a simplified block diagram illustrating a test system constructed and operative in accordance with a preferred embodiment of the present invention. The inspection system of FIG. 1 includes a sensor array 100, which typically includes a plurality of multi-pixel line sensors oriented such that their fields of view partially overlap and are arranged offset from each other, thereby requiring calibration. of. Typically, these multi-pixel line sensors are mounted in a camera with an electronic shutter such as a CCD camera.

A conveyor belt 102 passes an inspected item through the sensor array 100 in a conveying direction indicated by arrow 104 . Alternatively, a scan of the entire inspected item may also be provided by moving the sensor array 100 . A mapping function generator 106 is used to receive the output of the sensor array 100 while the sensor array 100 is detecting a test pattern 108 . Mapping function generator 106 provides a corrected output to correction circuit 110 which utilizes the mapping function generated by mapping function generator 106 . Circuitry 110 will receive the output of sensor array 100 when an inspected item such as printed circuit board 111 is being inspected by sensor array 100 .

It should be noted that, under normal circumstances, when a series of verification operations is started, in order for the mapping function generator 106 to be able to generate the information required to correct the output of the circuit 110 . The test pattern 108 needs to be verified with the sensor array 100 first. Multiple items to be inspected can then be inspected, however, thereafter in the middle of inspecting these items, the test pattern 108 can also be inspected again with the sensor array 100 to provide an updated calibration. In the case of continuous operation of the inspection system, the test pattern 108 is preferably inspected once a month.

Image correction circuit 110 can use the correction output received from mapping function generator 106 to correct the output received from sensor array 100 and provide a corrected sensor array output to segmentation circuit 112 . Segmentation circuitry 112 provides a segmentation output indication that typically classifies the overall region on the image of the item represented by the corrected sensor array output into one of two categories. For example, for the case of inspecting a printed circuit board, each location on the image represented by the corrected sensor array output will be differentiated by the segmentation output indication as either substrate or copper.

Segmentation output indications generated by segmentation circuitry 112 are provided to image processing circuitry 114 . Image processing circuit 114 is preferably a morphology based system, but could also be a bitmap, netlist, or any other suitable input based system. Circuitry 114 will provide an image processing output that specifies the various features of the image represented by the corrected sensor array output and the locations of those features. In the case of a printed circuit board, these features are typically pads, conductor connections, opens and other printed circuit board elements.

The image processed output of circuit 141 is provided to feature registration circuit 116 which will map the coordinate system of the image processed output of circuit 114 onto a feature reference coordinate system based on information provided by a reference input source 118 .

Both the output of registration circuit 116 and the output of reference input source 118 are provided to feature comparison circuit 120, which will compare the mapped circuit 114 image processing output with a reference stored in source 118 and provide a defect indication, The defect indication is in turn provided to a defect indication output generator 122 .

Referring now to FIG. 2, this is a simplified diagram illustrating various portions of a preferred test pattern 108, some of which are not drawn to scale, and simply shows the sensor array 100 "viewing" the test pattern 108 (Fig. 1) The field of view of each line sensor 124. Typically, the test pattern 108 includes a row of gratings 130 consisting of parallel, equally spaced slanted lines 132, an angular orientation determiner 134 with a slanted edge 182, and an array of LORs 136, preferably located at each The edge of the field of view of each camera 124. Among them, "LOR" stands for "many rectangles", which is used to indicate multiple rectangles of different sizes, as shown in the enlarged image coded 138 in the figure. Each LOR 136 is used to achieve relative positioning of an image of an object as "seen" by sensor array 100 (FIG. 1).

The equally spaced parallel inclined lines 132 preferably have a small included angle φ with a tangent value of about 0.05 with respect to the conveying direction 104 . The shape of the angular orientation determiner 134 is preferably close to a rectangle, but preferably has a small angle β with a tangent value of about 0.0156 between one of its sides and the conveying direction 104 .

Referring now to FIG. 3, there is a simplified block diagram illustrating mapping function generator 106 of FIG. An output 200 representing an image of an object on a test pattern 108 (FIGS. 1 and 2) being inspected by sensor array 100 (FIG. 1) is provided to mapping function generator 106, which will perform the following functions:

Perform "cell size and shape function determination" 202 for a single color;

"Test Pattern Angle Determination" 204;

"Sensor relative orientation determination" 206 by "image correlation of test pattern targets (preferably individual LORs 136 (FIG. 2))" at the "edge of sensor field of view";

"Sensor X Overlap and Y Offset Determination" 208 by "Neighboring Image Correlation"; and

"X and Y Offset Determination" 210 is performed for multiple colors.

The parameters thus determined are provided to a geometric polynomial generator at 212, which preferably provides a polynomial that maps the positions "seen" by elements of the sensor array 100 (FIG. 1) in at least two directions. function.

The output of geometric polynomial generator 212 is provided to image correction circuit 110 .

Referring now to FIGS. 4A and 4B, wherein FIG. 4A is a simplified flowchart illustrating the operation of the "pixel size and shape function determination" circuit 202 in FIG. Sketch.

It can be seen from FIG. 4B that in the field of view of the camera 124 with a given angle of view φ, the apparent size of the same figure "seen" by the camera 24 will be different depending on the position of the figure. Thus, as shown, an "on-axis" graphic 140 positioned directly in front of the camera and at an angle equal to the camera's field of view "appears" to have a width of d pixels, while an "on-axis" graphic 140 positioned at the edge of the field of view Similarly, the graphic 142 does not have an opening angle equal to the field of view of the camera, and the "seeing" width is d-Δ pixels.

Considering FIG. 4A now with reference to FIG. 4C, it can be seen that the grating rows 130 consisting of equally spaced parallel sloped lines 132 in the test pattern 108 (FIG. 2) can be “seen” by multiple cameras 124, FIG. 4C only showing One of the cameras came out. Each camera 124 will acquire an image of a portion of the row 130, a magnified view of which is shown at 150 in the figure. For computing size and cell shape functions, it is best to acquire images with a single color, such as red. Alternatively, you can acquire the image in several colors and use one of the colors to calculate the size and cell shape. For each image, the angle ψ of the image of line 132 with respect to transport direction 104 is measured. These line images are marked with code 152. The spacing between each pair of adjacent lines 132 in the row 130 is fixed and predetermined; The location of 105 is known. The location of a typical line is marked by code 154.

Graph 160 represents the sum of the outputs of individual cameras 124 as they scan row 130 along a direction at an angle ψ to axis 105 (ie, when the image is projected at ψ on the X axis). It can be seen that each line 152 produces a local minimum on the curve 160 . The spacing of adjacent lines 152 in the camera output can be determined by measuring the distance between adjacent inflection points 154' preceding each local minimum, where 154' represents the inflection point corresponding to line location 154. The X-axis of the graph 160 represents the number of each initial diode in a linear diode array of the camera 124 , and the Y-axis represents the sum of the intensity L of the image scanned along the direction 105 at an angle ψ.

Using the knowledge that the lines 132 in the test pattern are equidistant from each other, the change in position of the lines 152 can be mapped as a function of the number of diodes to indicate the presence of distortion in the camera 124 output image and the correction required. This mapping is given by graph 170, which is a least squares fit to a cubic function representing the size and shape functions, where the Y-axis is labeled "POS" and the "154" on it represents the line position 154 is the distance from any selected line 132 , and the X-axis represents the number of diodes in the camera 124 . This function can be expressed as

s _n (d)=s ₁ d+s ₂ d ² +s ₃ d ³

where n represents the camera number, s ₁ to s ₃ are three undetermined coefficients, and d is the diode number corresponding to the diode count of each camera n.

Referring now to FIGS. 5A and 5B, wherein FIG. 5A is a simplified flowchart illustrating the operation of the "test pattern angle determination" circuit 204 of FIG.

As shown in FIG. 5B , in practice, usually the entire test pattern target 108 is not perfectly aligned with the conveying direction 104 , but is offset by an angle α.

Referring now to FIG. 5A again with reference to FIG. 5C, the test pattern 108 is "viewed" by the camera 724 and includes an angular orientation determiner 134 having an edge 182 having a characteristic angle β with respect to other figures in the test pattern. . Camera 124 captures an image of angular orientation determiner 134, a magnified view of which is shown at 184 in FIG. 5C.

From the measurements for each scan line of the image, the characteristic angle β ^* of the slanted edge 186 of the image of the angular orientation determiner is calculated using conventional techniques. The calculated deviation of the characteristic angle β ^* relative to the angle β represents the value of the angle α, which can be expressed as β ^* -β=α. The offset thus determined is used in the calibration of circuit sum 110 (FIG. 1).

Referring now to FIGS. 6A and 6B , wherein FIG. 6A is a simplified flow diagram illustrating the operation of “relative sensor orientation determination” performed by the “test pattern target image correlation at edge of sensor field of view” circuit 206 in FIG. 3 , and FIG. 6B is an illustration. A diagram of the distortion required to be corrected by the function of circuit 206.

As shown in FIG. 6B , the fields of view 162 "seen" by the various cameras 124 are not aligned on a straight line or "collinear", but are offset from each other and partially overlap (exaggerated in the figure). This skew can be calculated using the angles θ ₁ to θ ₃ between the field of view axis of each camera and an axis 308 perpendicular to the motion direction 104 . The function to be explained here with reference to FIGS. 6A-6C is to correct the relative skew distortion of each camera’s field of view. The function to be explained below with reference to FIGS. X overlap and Y offset issues.

Considering FIG. 6A now with reference to FIG. 6C again, FIG. 6C shows a plurality of cameras 124 "viewing" a test pattern target, which preferably includes a row 300 of LORs 136 defined relative to angular The angular orientation of edge 182 of device 134 is known. The individual LORs 136 in the row 300 are preferably collinear, and the row 300 is preferably parallel to the front edge of the pattern target 108. Each camera 124 acquires an image of a portion of row 300 . The "seeing" LORs are preferably located at the edges of the respective fields of view of the respective cameras 124 .

An enlarged view of two image regions 253 "seen" by one of the cameras 124 (e.g., camera 1 (CAM1)) is shown in FIG. , and each contain a different LOR 136. It should be noted that due to the angle of the field of view of CAM1, there is an offset relationship between images 304 and 306, including a y offset, Δy _ANG shown in FIG. 6C. This offset is determined and used to calculate the angle θ ^* between the field axis of CAM1 and the LOR row 300 . The formula for calculating θ ^* is θ ^* =arctan(Δy _ANG /L _x ), where L _x is the longitudinal spacing between two adjacent LORs. The θ ^* angles for each camera 124 can be calculated in this way and stored for later use.

Referring now to FIGS. 7A and 7B , wherein FIG. 7A is a simplified flowchart illustrating the operation of "X overlap and Y offset determination of sensors" performed by "Neighboring Image Correlation" 208 in FIG. A schematic diagram of the function of circuit 208 to correct for distortion.

Similar to FIG. 6B , it can be seen from FIG. 7B that the fields of view 162 "seen" by the various cameras 124 are not aligned in a straight line or "collinear," but rather in the Y direction (here the same as direction 104), relative to each other. Skew and offset, with partial overlap (exaggerated here) in the X direction (here perpendicular to direction 104). The functionality described here with reference to FIGS. 7A-7C is to account for offsets in the fields of view of the cameras 124 .

Consider Fig. 7A again now with reference to Fig. 7C, Fig. 7C shows that a plurality of cameras 124 are "watching" test pattern target 108, and like Fig. Each adjacent camera 124 acquires an image of a portion of row 300 that includes the same LOR.

The X overlay and Y offset can be determined using an image field 253 captured by the two cameras 124 . Two enlarged images of an image region 253 containing one LOR 136 as "seen" by two adjacent cameras 124 are shown at reference numeral 354 in FIG. 7C. It can be seen that the two enlarged images of the area 253 are mutually offset and overlap each other, and the LORs in these two images are precisely overlapped and aligned. It should be noted that due to this overlapping relationship of the two images, a y offset Δy _ov and an x offset Δx _ov are generated, both of which are expressed in units of pixels. Δx _ov can then be converted to metric units, denoted mΔx _ov , using the cell size and shape functions previously described with reference to FIGS. 4A-4C . The amount of overlap OV _x in the x direction can be expressed as w-mΔx _ov , where w is the metric width of each image. Δy _ov can be converted to metric units by multiplying Δy _ov by a predetermined pixel size in the Y direction 104 .

Referring now to FIGS. 8A and 8B, wherein FIG. 8A is a simplified flowchart illustrating the operation of the "multicolor X and Y offset determination" circuit 210 of FIG. A sketch of the .

A three-color CCD camera 380 is shown in FIG. 8B. Typically, camera 380 includes a multi-row sensor 382 having three line sensor arrays (e.g., 384, 386, and 388), wherein each line sensor array is arranged parallel to each other, and each array includes a plurality of monochromatic sensors arranged in a line. Sense diode 390. The individual diodes 390 of the multiline sensor 382 can be logically divided into diode groups, each group consisting of three diodes belonging to the line sensor arrays 384, 386 and 388, each detecting a different color. Three such diode groups 392, 394 and 396 are drawn in the figure at the center and at the ends of the camera 380, respectively.

Camera 380 "views" cells 398 on an object 400 moving in direction 104 . Typically, the images acquired by the multi-line sensor 382 are stored in three buffers 402, 404 and 406 respectively, each buffer corresponding to a particular color. For example red, green and blue respectively. Symbol 408 represents a combined diagram of registers 402 , 404 and 406 . Combined buffer 408 shows the acquired images 399 , 401 and 403 of unit 398 . Images 399, 401 and 403 of combined buffer 408 show X-direction pixel size and shape distortion 410 due to chromatic aberration and spatial separation of R, G, B, (red, green, blue) line sensor arrays. The resulting offset 412 in the Y direction.

Referring now to FIG. 8A again with reference to FIG. 8C, FIG. 8C shows a plurality of cameras 124 "watching" the test pattern target 108 preferably containing a row 300 of LORs 136 as shown in FIGS. 6C and 7C. Each camera 124 preferably acquires a polychromatic image of one LOR 136 at either edge of its field of view and an image of an adjacent LOR 136 at the other edge of its field of view.

Preferably, each color of each multicolor image acquired by each camera at the edge of its field of view is correlated separately. A certain color (such as red) can be selected as a reference color, and the other two color components of the multicolor image will be compared with this reference color. In the example shown, a magnification of the red and green components of the image field 253 at one edge of the field of view of the camera CAM1 is shown at 360 . The two magnified images showing region 253 are in an offset and overlapping relationship with each other, while the two LORs are in precise overlapping alignment. It should be pointed out that due to this overlapping relationship between the two images, the y-shift Δy _COL will be caused by the different spatial positions of the line sensor arrays of different colors, and the x-shift Δx _COL will be caused by the color difference. The average shift is in units of pixels. [Delta]c _COL and [Delta]y _COL can then be converted to metric units using the same method as previously described with reference to Figure 7C. For the red and blue components of image region 253 at this edge of CAM1's field of view, and for the red, green and red and blue components of a polychromatic image of an adjacent LOR acquired by CAM1 at the other edge of its field of view, Comparisons can also be made in a similar manner (not shown).

Referring now to FIGS. 9A and 9B , which in combination form a simplified flowchart illustrating the operation of the geometric polynomial generator 212 of FIG. 3 . Using the outputs of 202-210 previously described in conjunction with FIGS. 4A-8C, the position of the diodes of each camera 124 can be determined by constructing a cubic function. Two sets of polynomials can be constructed for each camera 124: a set of X polynomials for determining the position of a diode in the X direction and a set of Y polynomials for determining the position in the Y direction. The X polynomial can be expressed as:

P _x (d)=a ₀ +a ₁ d+a ₂ d ² +a ₃ d ³

Where a ₀ -a ₃ are the respective coefficients of the X polynomial. The Y polynomial can be expressed as:

P _y (d)=b ₀ +b ₁ d

where b ₀ and b ₁ are the coefficients of the Y polynomial.

It has been found through experimentation that expressing the Y polynomial as linear provides sufficient approximation accuracy for the Y position of the diode.

The method of determining the X polynomial will now be described in more detail.

In FIG. 9A , first use the X overlap OV _x determined in 208 ( FIG. 3 ) to find a color (such as red ) of a ₀ [n]. The X polynomials for other colors (such as blue and green) are then derived from the X polynomials for the first color as follows:

1) Let a ₀ [1]=0 for CAM1

2) Determine a ₀ for each subsequent camera n along the camera line according to the following formula:

a ₀ [n]=a ₀ [n-1]+(S _n-1 (ND)-W)+OV _x [n,n-1]

in

- ND is the number of diodes in the previous camera n-1;

-S _n-1 (ND) is the value output by the pixel size and shape function estimated for the last pixel of camera n-1 in 202;

-W is the width in metric units of the image containing the LOR;

- OV _x is the amount of overlap in the X direction determined in 206 .

3) Designate the coefficients S1, S2, S3 represented by the size and shape function outputs determined by the circuit 202 (FIG. 3) as a ₁ , a ₂ , a ₃ of the X polynomial, respectively.

^Red is generated by combining the red-green and red-blue X offsets determined by circuit 210 (FIG. 3) and the X polynomial (a ₀ , a ₁ , a ₂ , a ₃ ) determined for the red component according to the following formula: a ₀ -a ₃ values of the green X polynomial:

a0[green]=a0[red]+(dr*rg_xl-dl*rg_xr)/(dr-dl)

a1[green]=a1[red]+(rg_xr-rg_xl)/(dr-dl)

a2[green]=a2[red]

a3[green]=a3[red]

in:

d _r is the diode location for the LOR that overlaps the right edge of the camera;

d _l is the metric location of the LOR for the left edge of the camera;

rg_xl is the Δx _COL difference between the red and green components measured at the left edge of the camera;

And, rg_xr is the Δx _COL difference measured at the right edge of the camera.

Except that the field edges of the first camera 1 and the last camera n are defined only for cameras n+1 and n-1, respectively, the "left" and "right" edges of the field of view for the other cameras refer to The camera edge closest to camera n-1 and the edge closest to camera n+1. The same procedure can be used to calculate a ₀ (blue) to a ₃ (blue).

The method of determining the Y polynomial will now be described in more detail with particular reference to FIG. 9B.

Referring again to FIG. 6B , for each camera 124 , an angle θ _1-3 between its field of view axis 162 and an axis 308 perpendicular to the direction of motion 104 may be determined. θ _1-3 can then be determined by the relationship θ = θ ^* -α, where α is the correction angle determined by circuit 204 ( FIG. 3 ).

In FIG. 9B, the Y polynomial is initially determined for one color (eg red) for each camera n in a camera row (eg cameras CAM1, CAM2, CAM3). The Y polynomials for other colors (such as blue and green) are then derived from the Y polynomials for the first color.

The coefficient b ₁ of the Y polynomial can be derived as b ₁ =tan(θ). The coefficient b ₀ of the Y polynomial can be calculated in two steps. The first step is to set _b0 of the first camera CAM1 to zero: _b0 [1]=0. The b ₀ of each subsequent camera can be calculated as follows:

b0[n]=b0[n-1]+b1[n-1]*(S _n-1 (ND)-OV _x )+Δy _ov [n,n-1]

In the second step, the minimum of the three Y polynomials for the three cameras is determined. Since it is a linear approximation, it is only necessary to find the minimum value at the edge of the field of view of each camera according to the following formula:

min(P _y1 (0),P _y1 ND),P _y2 (0),P _y2 (ND),P _y3 (0),P _y3 (ND),)

The subscript of each P _y represents the serial number of each camera. Subtracting this minimum value from b ₀ for each camera guarantees P _y ≥ 0 for all diodes.

By combining the red-green and red-blue y offsets Δy _col determined by circuit 210 (FIG. 3) and the Y polynomial determined for the red component, b ₀ and b ₁ values for the green and blue Y polynomials are generated. The specific formula is as follows:

b0[green]=b0[red]+0.5*(rg_ly+rg_ry)

b1[green]=b1[red]

where rg_ly is the Δy _COL offset between the red and green components measured at the left edge of the camera, and rg_ly is the Δy _COL offset measured at the right edge of the camera's field of view. As will be explained in more detail later with reference to Figure 17, _b0 [green] and _b0 [blue] are preferably modified to accommodate the different dawn times of each color component.

Referring now to FIGS. 10A and 10B , wherein FIG. 10A is a diagram illustrating a plurality of cameras 500 capturing images of an object 504 under ideal conditions, and FIG. 10B is a diagram illustrating an image buffer for storing acquired images of an object 504. picture. Typically, the positions of the cameras 500 and 502 are fixed, each has a static field of view, and the target 504 passes through the fields of view of the cameras 500 and 502 along the motion direction 104 . Cameras 500 and 502 each acquire images of target 504 one image line at a time using the aforementioned multi-pixel line sensors.

Each diode of the multi-pixel line sensor captures an image of a single pixel at a specific position on the target 504 , and the set of pixels captured by each diode constitutes an image row. An image line portion 512 comprising a plurality of pixels 514 is shown. As object 504 moves along motion direction 104 , the field of view of cameras 500 and 502 "moves" in the direction indicated by arrow 508 , so that each image row is acquired along direction 508 .

Note the time stamp 510 from t ₀ to t ₁ , at which camera ₅₀₀ begins acquiring images, producing one image row 516 represented by a dashed line. At time t _x , an image line portion 512 is acquired, which intersects a portion of an image cell 518 . As the target 504 continues to move in the direction of the arrow 506 , the image line captured at time close to t ₁ is the image line portion 520 shown by the dotted line.

The so-called ideal condition for the cameras 500 and 502 to acquire the image of the target 504 means that the fields of view of the two cameras are aligned on a straight line and do not overlap each other. As shown in FIG. 10A , at time t _x , image line portion 512 captured by camera 500 and image line portion 522 captured by camera 502 are aligned on a straight line, and image line portions 512 and 522 meet at boundary line 524 . catch.

Typically, the respective image lines scanned by cameras 500 and 502 are stored in buffers, such as buffers 530 and 532 in FIG. 10B . Since the scanlines in the two buffers are aligned as a straight line corresponding to the corresponding instant, an undistorted combined image of the target unit 518 can be formed by combining buffers 530 and 532, a portion of which is denoted by 534 said.

Referring now to FIGS. 11A and 11B, they illustrate that when the cameras 500 and 502 are not in the ideal conditions shown in FIGS. The effect obtained when overlapping conditions. Figures 11A and 11B are presented as extremely simplified illustrations of some of the difficulties actually encountered, merely to review what has been detailed above, and not to replace the illustrations of Figures 1-9B.

As can be seen from buffers 560 and 562 and image line portions 564 and 566 in FIGS. 11A and 11B , the image lines acquired by the two cameras at a given time _tx are not aligned in a straight line. As shown in FIG. 11B , combining buffers 560 and 562 results in a distorted combined image 568 of target unit 518 . Furthermore, simply combining the two buffer images does not correct for the image overlap discussed above with reference to Figures 7A-7C nor the perspective distortion discussed above with reference to Figures 4A-4C.

The technique for deriving a corrected combined image from buffers 560 and 562 will now be described with additional reference to FIGS. 12-16.

A FIFO (first-in-first-out) buffer can be defined by defining a window with a certain height (expressed by the fixed number of rows of the image buffer, such as 40) starting from the first obtained pixel row, such as in Fig. FIFO buffer 600 in 12. Then, whenever a new row of cells is fetched, the window is advanced one row to a new position. Alternatively, the image buffer can be initially filled with pixel rows, in which case the FIFO buffer window can be defined as a subset of image buffer rows, and the subset progresses row by row within the image buffer as described above .

Before the X and Y polynomials previously determined with reference to Figures 9A and 9B can be used to correct the image, they can be converted into another type of polynomial, which will be referred to herein as a "diode compensating polynomial". This polynomial will map a diode to a cell location in a rectified image constructed with a user-selected cell size. The two pipe compensation polynomials Q _x (d) and Q _y (d) can be derived from the X and Y polynomials using the following transformation:

Q _x (d)=px(d)/p

Q _y (d)=py(d)/p

where p is the pixel size selected by the user, in the same metric units as P _x (d). The pixel size p selected by the user must be an integer multiple of the minimum measurable distance along the travel in the scanning direction 104, which is typically one pulse unit of a drum encoder. The diode compensation polynomial _Qx also allows for additional adjustments for offsets caused by different integration times of the different color components, as will be described in more detail below with reference to FIG. 17 .

Each pixel or grid point in the FIFO buffer represents a sample of the corresponding target acquired by a diode. Perform a "resampling" process, so that the gray value g at any position between the grid points of the cache grid can be calculated, and the value of g can be interpolated by the following four-point convolution:

g=c ₁ g ₁ +c ₂ g ₂ +c ₃ g ₃ +c ₄ g ₄

Among them c ₁ to c ₄ are the convolution coefficients, g ₁ to g ₄ are the gray values of 4 adjacent grid points.

A method of determining the 4 interpolation coefficients c ₁ to c ₄ (total written as _ci ) and the 4 grayscale values g ₁ to g ₄ (total to be written as gi ₎ will now be described.

The 4 gray values g _i can be selected from the 4 connected grids of the FIFO buffer. These 4 points are referred to as a "4-point group" here. The method of determining which 4 grid points to select from the buffer grid will be described later.

Resampling can be performed in two stages corresponding to the aforementioned X and Y directions. These two stages are referred to here as X-resampling and Y-resampling. Y resampling will now be explained in more detail with reference to FIG. 12 . Two FIFO buffers 600 and 602 are shown corresponding to the two cameras. The entire group of picture elements 604 in the FIFO buffer scanned by one diode d may be referred to as a "gray column" of diode d, such as column 606 . The dashed scan lines 608 and 610 shown in the figure indicate for each buffer the correction angle required to compensate for the misalignment angle of the corresponding camera. A 4-point group 612 is shown, which includes the 4 pixels closest to the scan line 608 in the gray value column 606 .

Perform the following steps:

1. Compute the value of the resampling polynomial Q _y (d) for each diode d.

2. Use the 4-dot group indicator q(d) to indicate the first grid point in the 4-dot group in the gray-scale column of the diode. The other three grid points in this 4-point group are the first three grid points in the diode grayscale column, that is, the three latest acquired by diode d before acquiring the grid point indicating q(d) grid point. q(d) is determined by:

q(d)=floor(Qy(d)-1/2)-1

The indication q(d) for each 4-point group is preferably stored in a 4-point group look-up table at the location corresponding to diode d.

3. Calculate the four convolution coefficients c ₁ to c ₄ according to the distances between multiple multi-Q _y and the nearest 4-point group index. The above distance is called ξ(d) and can be expressed as:

ξ(d)=Qy(d)(q(d)+1)

It is best to have -1/2=<ξ<1/2

4. For a given ξ, four convolution coefficients C ₁ to C ₄ can be calculated as follows:

c ₁ =-14+10ξ-3ξ ² +0.5ξ ³

c ₂ =-1+0.5ξ+2ξ ² -1.5ξ ³

c ₃ =1-2.5ξ ² +1.5ξ ³

c ₄ =-5/2-5.5ξ-1.5ξ ² -0.5ξ ³

As mentioned earlier, ξ is related to d.

The convolution coefficient C is explained in more detail in Stephen K.Park and Robert A.Schowengerdt's paper "Image Reconstruction by Parametric Cubic Covolution" (Computer Vision Graphics and Image Processing, 23, 258-272). ₁ to C ₄ , the disclosure of this paper is hereby incorporated by reference.

For each diode d, the four coefficients _C1 to _C4 are preferably encoded such that when they are decoded and summed the resulting sum will equal 1, although the precision of any individual coefficient will be lost after encoding. The coded value is preferably stored in a coefficient look-up table at a location corresponding to d.

The overlay correction function will now be explained in more detail with reference to Figure 13, which shows the outputs 620 and 622 of two adjacent cameras 1 and 2 after Y resampling but before being combined. In order to combine the outputs of the two cameras into a single image, it is necessary to correct for image overlap 624 between the two cameras. In a preferred embodiment, instead of combining the two simply by using the output of camera 1 up to any selected pixel position within the overlapping region and then switching to the output of camera 2 starting from a corresponding pixel position image output. The method adopted in this embodiment is to first define a "mixed area" 626 containing a predetermined number of pixels B (typically 100 pixels) in the overlapping area 624 between cameras 1 and 2, and then define a "mixed area" 626 for the mixed area Corresponding pixels of the two cameras are blended to obtain a single pixel value, which is then used to form a combined image 630 . In order to accommodate the low quality output often encountered at both ends of the diode array, edge regions 628 containing a predetermined number M of pixels (typically 20 pixels) are preferably left in the overlap region on both sides of the mixing region.

It can be seen that the two diode resampling polynomials Q _x ⁽¹⁾ (d) and Q _x ⁽²⁾ (d) of two adjacent cameras.

Can be used to determine the amount of overlap between two cameras.

In order to correct for overlap, the following steps can be performed:

1. Define Q _x ⁽²⁾ (1) to represent the pixel position r of the first pixel of the X and Y resampled output of camera 2 (X sampling will be explained in more detail with reference to FIG. 15 later). The leading edge of the mixing zone can be determined by adding the predetermined number of pixels of the aforementioned edge zone.

2. As shown in FIG. 14 , determine a weight w(i) for each position i in the mixed area, where w(i)=i/B. Using this weight allows for a linear blending of the outputs of cameras 1 and 2 when combining the outputs of these two adjacent cameras, where the contribution of camera 1 will be 1-w(i) and the contribution of camera 2 will be w(i ). For example, when the mixed zone consists of 100 pixels, the first one of them will contain 99% of the information of the camera 1 pixel and 1% of the corresponding camera 2's information, and the last one in the mixed zone Cells have the opposite scale. This solution enables a smooth transition between the two cameras.

3. Write the grayscale output of the pixels corresponding to cameras 1 and 2 in the mixed area as:

g(i)=(1-W(i))*g1(i)+w(i)*g2(i)

where B is the number of pixels in the mixed area, i is the location index in the area, g ₁ (i) and g ₂ (i) are the gray values of the corresponding pixels in the mixed area of cameras 1 and 2, respectively.

X-sampling will now be described in more detail with reference to FIG. 15 . Due to optical distortions and to fit the user-defined pixel size, it is necessary to resample the Y sampled output 640 in the X direction and thereby generate an X corrected image row 642 with a pixel of the desired size. The position r of each pixel on the X-corrected image line corresponds to a position d(r) on the diode array.

d(r) can be calculated using the diode resampling polynomial Q _x (d). This will involve finding the inverse function Q ^-1 (r) of Q _x (d). This inverse function will map the pixel location on the X-corrected image line to a corresponding location d(r) on the diode array. It can be seen that this position may not correspond to the integer-valued position of a particular diode, but may need to be expressed as a non-integer value counting along the diode array. Once the diode position _dp is found, a "X-pixel 4-point group" corresponding to 4 diodes can be determined in a manner similar to that described above for Y resampling. Then the gray value at position r on the X-corrected image line is interpolated by convolving the gray value of these 4 pixels with 4 correlation coefficients.

In the following, the indication q(r) is used to indicate the position of a 4-point group currently being processed.

Referring now again to Figure 16, this figure illustrates some of the steps that need to be performed for each pixel in the X-corrected image row:

1. The specified index r _p corresponds to the first pixel 650 in an X-corrected image row 652 .

2. For each pixel position corresponding to the overlapping area 654 in the X-corrected image row 652, gradually increase the value of the indicator _rp until the last field of view 656 of the current camera CAM1 field of view, and find out the diode that satisfies the following formula Position d _p ⁽¹⁾ so that r _p ⁽¹⁾ =Qx(d _p ⁽¹⁾ ) Since Q _x is a monotonous increasing function, d _p will increase with the increase of r _p . When d _p reaches the end of the field of view 656 of CAM1, r _p will return to the pixel position corresponding to the beginning of the overlap region 658 of the next camera CAM2, and assign the value of r _p p equal to the first corresponding camera CAM2 The diode compensation polynomial estimated by the diode, ie, r _p ←Q _x ⁽²⁾ (1).

3. For each pixel position in the X correction image row 652 of CAM2, gradually increase the index r _p , and find out the diode position d _p ⁽²⁾ satisfying the following formula so that r _p ⁽²⁾ =Qx(d _p ^{( 2)} ), and then go to step 2.

For each subsequent pair of cameras, steps 2 and 3 are performed.

When obtaining the diode position d _p , it can be completed by obtaining the inverse function of the function Q _x (d _p ) at one time, or it can be completed by numerical solution. Once the diode position d _p is obtained, the index and ζ of the first pixel in the X-4 point group can be expressed as follows:

q(r)=FLLOR(Q ^-1 _x (r)-1/2)-1

ξ(r)=Q ^-1 _x (r)-(q(r)+1)

This is similar to the case of Y sampling.

Thus, q(r) defines the X-4 point group, and the convolution coefficients C ₁ to C ₄ can be calculated from ζ using the formula given above for Y resampling.

4. Store q(r) in an X-4 point group lookup table at the location corresponding to pixel location r. Alternatively, calculate the offset from the q(r-1) position by subtracting the previous 4-point group position q(r-1) from the current 4-point group position q(r), then store the offset displacement.

5. Encode the convolution coefficients c ₁ to c ₄ using the method described above for Y sampling, and store the encoding results in an X coefficient lookup table corresponding to the pixel position r _p .

Image correction processing including Y resampling, X resampling and overlay correction is preferably performed by circuit 110 of FIG. 1, as summarized below.

In the Y resampling stage, image correction for misalignment of the camera field of view can be done as follows. For a given FIFO window position, the 4-point group for each diode is found from the Y-4-point group lookup table. Then four gray values g ₁ to g ₄ corresponding to the 4-point group can be extracted from the FIFO buffer, and four correlation coefficients c ₁ to c ₄ can be extracted from the Y coefficient lookup table. The final interpolated gray value for each diode can then be calculated as described above as follows:

g=c ₁ ^* g ₁ +c ₂ ^* g _{2 +c 3} ^* g _{3 +} c ₄ ^* g ₄

And store it in a Y-corrected grayscale buffer.

In the X resampling stage, the cell shape and size are corrected by further processing the output of the Y samples. The grayscale values of one pixel row are processed at a time. For each pixel in the X-corrected image row, the X-4 point group index is extracted from the X-4 point group lookup table, and four convolution coefficients C ₁ to C ₄ are extracted from the X coefficient lookup table. Then the 4 grayscale values g _{1 ′} to g ₄ ′ corresponding to the X-4 point group can be extracted from the Y correction grayscale buffer. Thereafter, the final interpolated gray value for each pixel position r _p can be calculated as described above:

g'=c' ₁ ^* g ₁ +c' ₂ ^* g ₂ +c' ₃ , ^* g ₃ +c' ₄ ^* g ₄

In the overlay correction stage, a single image line is formed by combining the X-resampled image line outputs of the individual cameras as explained above with reference to FIG. 16 .

It is well known that in color image acquisition systems the nature of the light source of the photographic target and the spectral reflectance properties of the target both contribute to non-uniform color intensity in the resulting image. So, for example, a red-detecting diode in a multi-row sensor array might receive a different amount of light than a blue-detecting diode for a white area on a target. Non-uniform color intensities can be corrected for by varying the accumulation time for each color row in an inversely proportional relationship to the light intensity received by that color row in a multi-row sensor array.

In the present invention, when starting to acquire an image line of an object, all electronic shutters of the camera are open, so each color line of the multi-line sensor array starts accumulating charge corresponding to its respective color. The accumulation time of each color row is then changed by closing the electronic shutters of each color row of the multi-row sensor array at different moments. However, the center of the cell obtained for each color row may not be the same as the geometric center of the cell. Thus when measuring the amount of overlap in the Y direction 104 as shown in FIG. 8C, an "accumulated offset" _Δyacc will result, which can be subtracted from the pixel geometric center of each color component by the following formula The center of the obtained cell is corrected:

Δ _acc [Green]=(AT[Green]-AT[Red])/(2*IT)

b0 ^* [Green]=b0[Green]+Δ _acc [Green]

where AT represents the accumulation time, IT represents the integration time (i.e. the time between the start instants of two successive image lines), and b ₀ ^* [green] is the modified zero-order coefficient of Q _y . The blue coefficients can be similarly corrected.

This cumulative offset is preferably determined when the test target is acquired and used to adjust the b ₀ coefficients of the Y polynomial Py. The diode compensation polynomial Q _y illustrated in FIG. 9B can also be used to adjust the accumulation offset by taking different integration times for different colors.

Figure 17 illustrates the cumulative offset problem in more detail. As previously explained with reference to FIG. 8C , there is shown a multi-row sensor array 700 acquiring three pixels 702, 704, and 706, which are acquired by different sensors 708, 710, and 712, respectively. Both include multiple single-color sensing diodes. Due to the different accumulation times of each line sensor, the three pixels acquired have different relative areas, as shown by the accumulation areas 714 , 716 and 718 . For these three pixels, it can be defined that their geometric centers are located at 720, 722 and 724, respectively. And their cumulative area centers can be defined at 726, 728 and 730, respectively. The distances 732, 734, and 736 between the cumulative area center and the geometric center of each pixel represent the cumulative offset of each color component and can be used to correct for overlap in the Y direction 104 as previously described.

It should be noted that, although for clarity purposes various features of the invention have been described in terms of separate embodiments, these may also be combined in a single embodiment. Conversely, although several features of the invention have been described in terms of a single embodiment for brevity, these features may also be presented individually or in any suitable subcombination in separate embodiments.

It should be pointed out that for those skilled in the art, the present invention is not limited to the contents specifically shown and described above. On the contrary, the scope of the present invention should include the combination and partial combination of the above-mentioned various features, and also include modifications and changes to these features that those skilled in the art may make after reading the foregoing description.

Claims (32)

1, a kind of image-taking system comprises:

A plurality of transducers that contain a plurality of sensor units separately;

A prescan calibration subsystem, it utilizes a presumptive test pattern of being surveyed by a plurality of transducers that an output indication about the distortion on the two-dimensional direction in the above-mentioned a plurality of transducers output is provided, and above-mentioned output indication is used to produce the function of the position that above-mentioned each the sensing unit institute that can shine upon on the two-dimensional direction at least " sees "; And

A distortion correction subsystem, it can utilize above-mentioned output to indicate when article of above-mentioned a plurality of sensor scans and proofread and correct above-mentioned distortion.

2, according to the image-taking system of claim 1, wherein above-mentioned a plurality of transducers comprise a plurality of transducers with different spectral sensitivities.

3, according to the image-taking system of claim 2, wherein above-mentioned a plurality of transducers comprise at least two and have the transducer of identical spectra sensitivity substantially.

4, according to the image-taking system of claim 1, it is partly overlapping transducer at least at least one direction that wherein above-mentioned a plurality of transducers comprise at least two.

5, according to the image-taking system of claim 1, wherein above-mentioned prescan calibration subsystem can be with the precision work of inferior pixel.

6, according to the image-taking system of claim 1, wherein above-mentioned distortion correction subsystem is carried out the non-zero order interpolation of each pixel in the output of above-mentioned a plurality of transducers.

7, according to the image-taking system of claim 1, wherein above-mentioned distortion correction subsystem can compensate the variation of the pixel dimension in above-mentioned a plurality of transducer.

8, according to the image-taking system of claim 1, wherein above-mentioned distortion correction subsystem can be repaid the magnification change in above-mentioned a plurality of transducer.

9, according to the image-taking system of claim 1, wherein above-mentioned distortion correction subsystem can compensate the aberration in above-mentioned a plurality of transducer.

10, according to the image-taking system of claim 1, wherein above-mentioned a plurality of transducers comprise the transducer that some have different spectral sensitivities, and wherein above-mentioned function is relevant with the above-mentioned different accumulated times that transducer adopted with different spectral sensitivities.

11, according to the image-taking system of claim 1, wherein above-mentioned distortion correction subsystem can compensate the variation of pixel shape in above-mentioned a plurality of transducer.

12, according to the image-taking system of claim 1, wherein above-mentioned distortion correction subsystem can be with 5% precision work of the pixel dimension that is better than above-mentioned a plurality of sensor units.

13, a kind of image-taking system comprises:

A plurality of transducers that contain a plurality of sensor units separately;

A prescan calibration subsystem, it utilizes a presumptive test pattern of being surveyed by a plurality of transducers that an output indication about the distortion in above-mentioned a plurality of transducer outputs is provided, and above-mentioned output indication output is used to produce a correction function; And

A distortion correction subsystem, it can utilize above-mentioned correction function to proofread and correct above-mentioned distortion when article of above-mentioned a plurality of sensor scans, and above-mentioned distortion correction subsystem can be with 5% precision work of the pixel dimension that is better than above-mentioned a plurality of sensor units.

14, a kind of image-taking system comprises:

A plurality of transducers;

A prescan calibration subsystem, it utilizes one to provide an output indication about the distortion in the output of above-mentioned a plurality of transducers at the presumptive test pattern of being surveyed by a plurality of transducers in respect to a plurality of sensor movements along a direction of relative movement, and each image of at least one target on the above-mentioned test pattern that above-mentioned prescan calibration subsystem can " be seen " above-mentioned a plurality of transducer institute is correlated with to determine the relative bearing of above-mentioned a plurality of transducers; And

A distortion correction subsystem, it can utilize above-mentioned output to indicate and proofread and correct above-mentioned distortion.

15, according to the image-taking system of claim 14, wherein above-mentioned prescan calibration subsystem can also provide one about the output indication of above-mentioned a plurality of transducers with respect to the orientation of above-mentioned scanning direction.

16, according to the image-taking system of claim 14, each transducer in wherein above-mentioned a plurality of transducers all contains a plurality of sensor units; And

Above-mentioned prescan is calibrated the pixel dimension characteristic that subsystem can also be determined each sensor unit in a plurality of sensor units of each transducer in above-mentioned a plurality of transducer.

17, according to the image-taking system of claim 16, wherein above-mentioned prescan calibration subsystem goes " watching " to be formed on the pixel dimension characteristic that the grating of being made up of a plurality of parallel equidistant lines on the above-mentioned test pattern is determined each sensor unit in a plurality of sensing units of each transducer in above-mentioned a plurality of transducer by making above-mentioned a plurality of transducer.

18, a kind of image-taking system comprises:

A plurality of transducers that contain a plurality of sensor units separately;

A prescan calibration subsystem, it utilizes a presumptive test pattern of being surveyed by a plurality of transducers that an output indication about the distortion on the two-dimensional direction in the above-mentioned a plurality of transducers output is provided, and above-mentioned output indication is used to produce the function of the position that the sensor institute that can shine upon on the two-dimensional direction at least " sees "; And

A distortion correction subsystem is used for utilizing above-mentioned output to indicate and proofreaies and correct above-mentioned distortion when article of above-mentioned a plurality of sensor scans.

19, according to the image-taking system of claim 18, wherein above-mentioned distortion correction subsystem can utilize an optional pixel dimension of user to come work.

20, a kind of item verification system comprises:

An image acquisition subsystem is used to obtain the image of article to be checked;

A graphical analysis subsystem is used for identifying from above-mentioned image at least one predetermined properties of this article; And

An output indication subsystem is used to provide an output indication about above-mentioned at least one predetermined properties of having article;

It is characterized in that above-mentioned photomoduel comprises:

A plurality of sensor clusters;

Self calibration equipment is used for the geometrical relationship between definite above-mentioned each sensor cluster; And

Transducer output corrective, be used for revising according to the above-mentioned geometrical relationship between above-mentioned each sensor cluster the output of above-mentioned a plurality of sensor clusters, the sensor output corrective comprises the electronics interpolation apparatus that can carry out the non-zero order interpolation to each pixel in the output of above-mentioned a plurality of sensory packages.

21, a kind of item verification system comprises:

An image acquisition subsystem is used to obtain the image of an inspected object;

A graphical analysis subsystem is used for identifying from above-mentioned image at least one predetermined properties of this article; And

An output indication subsystem is used to provide an output indication about above-mentioned at least one predetermined properties of having these article;

It is characterized in that above-mentioned photomoduel comprises:

A plurality of sensor clusters;

Self calibration equipment is used for the geometrical relationship between definite above-mentioned each sensor cluster; And

Transducer output corrective, be used for revising according to the above-mentioned geometrical relationship between above-mentioned each sensor cluster the output of above-mentioned a plurality of sensor clusters, the sensor output corrective can be adapted to the output of above-mentioned a plurality of sensor clusters the precision of inferior pixel.

22, a kind of item verification system comprises:

An image acquisition subsystem is used to obtain the image of article to be checked;

A graphical analysis subsystem is used for identifying from above-mentioned image at least one predetermined properties of this article; And

An output indication subsystem is used to provide an output indication about above-mentioned at least one predetermined properties of having these article;

It is characterized in that above-mentioned photomoduel comprises:

At least one sensor cluster; And

Transducer output corrective is used for revising according to the optical distortion relevant with above-mentioned at least one sensor cluster at least in part at least one output of above-mentioned at least one sensory package.

23, according to the item verification system of claim 22, wherein above-mentioned optical distortion comprises the pixel dimension distortion.

24, according to the item verification system of claim 22, wherein above-mentioned optical distortion comprises the magnification ratio distortion.

25, according to the item verification system of claim 22, wherein above-mentioned optical distortion comprises aberration.

26, according to the item verification system of claim 22, wherein above-mentioned optical distortion comprises overlapping mismatch.

27, according to the item verification system of claim 22, wherein above-mentioned optical distortion comprises because transducer separates the pixel skew that causes.

28, according to the item verification system of claim 22, the focusing that wherein above-mentioned optical distortion comprises each color composition is inconsistent.

29, according to the item verification system of claim 22, wherein above-mentioned optical distortion comprises color accumulation skew.

30, a kind of item verification system comprises:

An image acquisition subsystem is used to obtain the image of article to be checked;

A graphical analysis subsystem is used for identifying from above-mentioned image at least one predetermined properties of this article; And

An output indication subsystem is used to provide an output indication about above-mentioned at least one predetermined properties of having these article;

It is characterized in that above-mentioned photomoduel comprises:

At least one sensor cluster;

Transducer output corrective, be used to revise at least one output of above-mentioned at least one sensor cluster, the sensor output corrective comprises that one is used to produce an energy the function generator of the function of the location map on the sensor assembly to one group of scanning position.

31, a kind of item verification system comprises:

A photomoduel is used to obtain the image of article to be checked;

A graphical analysis subsystem is used for identifying from above-mentioned image at least one predetermined properties of this article; And

An output indication subsystem is used to provide an output indication about at least one predetermined properties of having these article;

It is characterized in that above-mentioned photomoduel comprises:

One makes the user can select user interface by the resolution of image that photomoduel obtains;

An electro-optical sensor assembly; And

An electronic resolution modifier that is used for above-mentioned electro-optical sensor assembly downstream.

32, according to the item verification system of claim 31, photomoduel wherein can select to determine the pixel dimension of above-mentioned image according to the resolution on the above-mentioned user interface.

Images