JP2013113600A - Stereo three-dimensional measuring apparatus - Google Patents

Abstract

Calibration deviation is corrected again without increasing the circuit scale and calculation time.
The amount of deviation between the left and right camera stereo-parallelized images corrected by the calibration is corrected during a corresponding point search process by a corresponding point searching unit including a calibration deviation re-correcting unit. Re-correction is performed using data obtained by the calibration deviation re-correction processing by the re-correction unit 15. Thus, the measurement accuracy can be increased without increasing the circuit scale and calculation time of the internal hardware in the distance measuring apparatus 1. In addition, the right camera stereo parallelized image is shifted up and down to recorrect the vertical shift of the left and right camera stereo parallelized images. In this way, it is possible to reduce the vertical calibration deviation that is likely to be a problem. At that time, it is possible to reliably find the optimal position of the right camera stereo parallelized image when searching for the corresponding point.
[Selection] Figure 1

Description

  The present invention relates to a stereo three-dimensional measurement apparatus that measures distance, shape, and the like using a plurality of cameras.

  A stereo three-dimensional measuring apparatus that measures a distance to an object and a shape of the object using a plurality of cameras has been put into practical use. As a method of stereo three-dimensional measurement, a method generally called “correlation method” is known.

  In the “correlation method”, cameras are arranged on the left and right, and an object is photographed at the same time, and a corresponding point that is a set of corresponding pixels is searched from the obtained pair of left and right images. Then, a parallax representing how far the corresponding points of the left and right images are separated in the left-right direction can be obtained, and the distance to the object can be calculated from this parallax by applying the principle of triangulation. In addition, there is a method called “light cutting method” in which a known light pattern such as a straight line or a polygon is projected onto an object, and the shape of the object is obtained from a change in the projected pattern shape. In some cases, measurement is performed in combination with "."

  By the way, when performing stereo three-dimensional measurement using a plurality of cameras, measurement errors occur due to error factors such as lens distortion, image sensor center misalignment, and left and right camera positional misalignment. In order to correct this, an operation called “calibration” is required before measurement.

  That is, a plurality of subjects called `` calibration boards '' printed with geometric patterns with known shapes are photographed at different distances and different angles, and calibration parameters including information on error factors are calculated from the image information. Keep it. Thereafter, when performing stereo measurement, a calibration parameter is applied to the captured left and right images to generate a corrected left and right image pair from which an error factor has been removed. By searching for corresponding points and calculating parallax using the corrected left and right image pairs, measurement with less error can be performed.

  However, in actual three-dimensional measurement, a change may occur in the above-described error factor due to a change over time during measurement, and there may be a phenomenon in which the measurement error gradually increases due to deviation from the calibration parameter. . To prevent this, calibration may be performed before each measurement, or calibration may be performed by pausing the measurement at regular intervals. It will greatly impair the sex.

  Therefore, in order to correct the calibration deviation caused by the calibration parameter deviation as described above, a stereo camera equipped with the calibration deviation correction apparatus disclosed in Japanese Patent Application Laid-Open No. 2004-354257 (Patent Document 1) is provided. Proposed.

  In the conventional stereo camera, a feature having a known geometric shape is extracted from the stereo image captured by the image capturing device by the feature extracting device, and the calibration data correcting device performs calibration based on the extracted feature. The calibration deviation relating to the calibration parameters stored in the calibration data recording apparatus is corrected.

  However, the conventional stereo camera has the following problems. That is, in addition to the corresponding point search unit required for normal three-dimensional measurement, the feature extraction device for detecting the feature points is required separately. Therefore, when the feature extraction apparatus is implemented by hardware, the circuit scale is increased. On the other hand, when the feature extraction apparatus is implemented as software, the calculation time is increased.

  The feature extraction by the feature extraction device is relatively easy when the rough shape of the measurement object is known in advance, but the measurement image has a low contrast or the shape of the measurement object is frequent. In such a case, there is a problem that application becomes very difficult.

  Furthermore, in recent years, an image sensor used for stereo three-dimensional measurement has been increased in the number of pixels and the pitch of the pixel has been reduced, and the demand for more accurate measurement has increased. When the pixel pitch of the image sensor is narrowed, the resolution of the measurement distance is increased in proportion thereto, so that highly accurate measurement can be performed. However, on the other hand, there is a problem that accuracy required for calibration is also increased.

JP 2004-354257 A

  SUMMARY OF THE INVENTION An object of the present invention is to provide a stereo three-dimensional measurement apparatus that can easily re-calibrate a calibration shift without causing an increase in circuit scale and calculation time and perform highly accurate three-dimensional measurement. is there.

In order to solve the above problems, the stereo three-dimensional measuring apparatus of the present invention is
A plurality of cameras that take images of measurement objects and acquire images;
A calibration calculation unit that calculates a calibration parameter by performing camera calibration using captured images of a plurality of camera calibration boards acquired by the plurality of cameras;
A storage unit for storing the calculated calibration parameter;
A stereo parallelization process is performed to correct an error including at least one of distortion, inclination, and shift included in the images of the plurality of measurement objects acquired by the plurality of cameras using the calibration parameter. Stereo parallelization processing,
A calibration deviation recorrector that recorrects the shift between the plurality of images after the stereo parallelization processing using data obtained by searching for pixels corresponding to each other in the images after the stereo parallelization processing; ,
The pixels corresponding to each other in the images of the plurality of measurement objects recorrected by the calibration deviation recorrection unit are searched as corresponding points, and the distances between the searched corresponding points are the plurality of the plurality of corresponding points. A corresponding point search unit that generates disparity information indicating a distance in a direction in which at least two of the cameras are arranged;
And a distance information generation unit for obtaining distance information from the parallax information generated by the corresponding point search unit to the measurement target.

  According to the above configuration, the calibration deviation re-correction unit searches for pixels corresponding to each other in each image after the stereo parallelization process for the deviation between the plurality of images after the stereo parallelization process by the stereo parallelization process. The data obtained in this way is used for recorrection. Therefore, when the error included in the image of the measurement object is corrected by the stereo parallelization process using the calibration parameter calculated by the calibration calculation unit, the deviation amount that cannot be corrected between the plurality of images is mutually corrected. Re-correction can be performed using data obtained by searching the corresponding pixel.

  Therefore, calibration deviation can be easily re-corrected and highly accurate three-dimensional measurement can be performed without increasing the circuit scale of internal hardware and the calculation time of the stereo three-dimensional measuring apparatus.

Moreover, in the stereo three-dimensional measurement apparatus of one embodiment,
When the direction in which the two cameras are arranged is the x direction and the direction orthogonal to the x direction in the plane of the image captured by the two cameras is the y direction,
The calibration shift recorrection unit recorrects at least the shift in the y direction between the images after the stereo parallelization process.

  According to this embodiment, the calibration deviation recorrection unit recorrects at least the deviation in the y direction between the images after the stereo parallelization processing. Therefore, the calibration deviation in the y direction (vertical direction), which is likely to be a problem when performing three-dimensional measurement based on images of a plurality of measurement objects acquired by a plurality of cameras having a high pixel count and a narrow pixel pitch, in particular. Can be reduced.

Moreover, in the stereo three-dimensional measurement apparatus of one embodiment,
A projector for projecting a two-dimensional random texture onto the measurement object is provided.

  According to this embodiment, a two-dimensional random texture is projected onto the measurement object. Therefore, even if the texture change on the surface of the measurement object is poor or the contrast is low, the corresponding point search unit can search for the corresponding point by applying a luminance change to the surface of the measurement object. It becomes easy and can perform three-dimensional measurement stably.

Moreover, in the stereo three-dimensional measurement apparatus of one embodiment,
The calibration deviation re-correction unit sequentially selects image pairs each including a reference image serving as a reference and a remaining comparison image compared with the reference image from the plurality of images after the stereo parallelization processing. A search parameter as the data is obtained by searching for a corresponding pixel in both images while shifting the comparison image of the selected image pair in the y direction with respect to the reference image, and based on the obtained search parameter Then, an optimum shift position of the comparison image is obtained, and the shift between the plurality of images is recorrected by shifting the comparison image in the y direction to the optimum shift position with respect to the reference image.

  According to this embodiment, in the image pair selected from the plurality of images after the stereo parallelization processing by the calibration deviation re-correction unit, the comparison images are shifted from each other in the y direction with respect to the reference image. Based on search parameters obtained by searching for corresponding pixels, the optimum shift position of the comparison image is obtained. Therefore, the optimum shift position of the comparative image can be found with certainty.

Moreover, in the stereo three-dimensional measurement apparatus of one embodiment,
The storage unit also stores the optimal shift position obtained by the calibration shift recorrection unit,
The calibration deviation re-correction unit, when re-correcting the deviation between the plurality of images, uses the optimum shift position stored in the storage unit to compare the comparison image with the reference image. Thus, the y-direction is shifted to the optimum shift position.

  According to this embodiment, the optimum shift position obtained by the calibration deviation re-correction unit is stored in the storage unit. Therefore, when the corresponding point search unit resumes the search for the corresponding point and the generation of the parallax image once interrupted, the calibration shift recorrection unit stores the optimal shift position stored in the storage unit. By using, the shift between the plurality of images can be quickly corrected again to the same position as before the interruption, and the corresponding point search unit can be quickly restarted.

Moreover, in the stereo three-dimensional measurement apparatus of one embodiment,
The corresponding point search unit monitors the search parameter when performing the corresponding point search and the generation of the disparity information, and when the value of the search parameter deteriorates beyond a predetermined value beyond a predetermined value. Instructs the calibration deviation recorrection unit to reacquire the optimum shift position.

  According to this embodiment, when the value of the search parameter has deteriorated beyond a predetermined value beyond the optimum value, the corresponding point search unit determines the optimum point for the calibration deviation re-correction unit. The re-acquisition of the shift position is instructed. Therefore, even when the corresponding point search unit continuously searches for the corresponding points and generates the parallax information, generation of measurement errors can be minimized.

Moreover, in the stereo three-dimensional measurement apparatus of one embodiment,
A display unit that displays at least the distance information generated by the distance information generation unit;
The calibration deviation re-correction unit, when the obtained values of the plurality of search parameters are less than a preset setting value, information that prompts the calibration calculation unit to execute the camera calibration It is designed to be displayed on the display unit.

  According to this embodiment, when the values of the plurality of search parameters obtained by the calibration deviation recorrecting unit are less than preset values, the calibration is performed on the user. Is encouraged. Therefore, it is possible to deal with an image shift that cannot be recorrected by the calibration shift recorrector, and to prevent an increase in three-dimensional measurement error.

  As is clear from the above, the stereo three-dimensional measurement apparatus according to the present invention uses the calibration deviation re-correction unit to correct the deviation between the plurality of images after the stereo parallelization process by the stereo parallelization process after the stereo parallelization process. Because correction is performed using data obtained by searching for pixels that correspond to each other in each of the images, even if multiple images of the measurement target are corrected by stereo parallelization processing using calibration parameters, correction can be completed. This amount can be re-corrected using the data obtained by searching for the corresponding pixels.

  Therefore, calibration deviation can be easily re-corrected without causing an increase in the circuit scale of internal hardware and an increase in calculation time in the stereo three-dimensional measurement apparatus, and highly accurate three-dimensional measurement can be performed.

It is a block diagram of the distance measuring device which is an example of the stereo three-dimensional measuring device of this invention. It is a flowchart of the three-dimensional measurement process by the distance measuring device shown in FIG. It is explanatory drawing of the stereo parallelization process and corresponding point search process in FIG. It is a figure which shows an example of the pattern used for a calibration board. It is a figure which shows the pixel shift of the up-down direction of the image after left-right camera stereo parallelization. It is a flowchart of a corresponding point search process including a calibration deviation re-correction process. It is a figure which shows the relationship between the up-down direction shift | offset | difference adjustment amount and the recognition rate at the time of calibration deviation | shift re-correction processing. It is a figure which shows the change of the pixel shift amount to the downward direction of the image after right camera stereo parallelization, and the change of a parallax image. It is a block diagram of the distance measuring device different from FIG. It is a flowchart of the character recognition process by the distance measuring device shown in FIG. It is a figure which shows an example of the two-dimensional random texture projected by the projector in FIG. It is explanatory drawing of the brightness | luminance change provided to the base surface and the character upper surface on which the two-dimensional random texture was projected.

  Hereinafter, the present invention will be described in detail with reference to the illustrated embodiments.

First Embodiment FIG. 1 is a block diagram of a distance measuring device as a stereo three-dimensional measuring device of the present embodiment. In FIG. 1, a distance measuring device 1 includes a stereo camera 2, a measurement calculation unit 3, a display unit 4, and a housing (not shown). Then, a distance image as the distance information representing the distance to the measurement object 5 is generated and displayed on the display unit 4.

  The stereo camera 2 is composed of a pair of left camera 6 and right camera 7. Here, the left camera 6 and the right camera 7 have a constant distance between the left camera 6 and the right camera 7 (that is, a base line length) and an angle between the left camera 6 and the right camera 7 (that is, a convergence angle). It is mechanically fixed inside the casing so as to have a value.

  In the distance measuring apparatus 1, the optical axis center of the image sensor in the left camera 6 is set as the origin, and the direction connecting the optical axis centers of the two image sensors in the left camera 6 and the right camera 7 is the x direction (left and right direction), left The direction orthogonal to the x direction in the image sensor surface of the camera 6 is defined as the y direction (up and down direction), and the x direction and the direction orthogonal to the y direction are defined as the z direction (depth direction). Each of the left camera 6 and the right camera 7 is a camera module in which a CMOS (complementary metal oxide semiconductor) imaging device, a condensing lens, and a control DSP (Digital Signal Processor) are integrated. Yes, by providing a control signal from the outside to the DSP, it is possible to electrically control camera parameters such as a photographed image size, focal length, exposure, and white balance.

  The measurement calculation unit 3 includes a stereo parallelization unit 8, a corresponding point search unit 9, a calibration calculation unit 10, a calibration parameter storage unit 11, a CPU (central processing unit) 12, and a distance image as the distance information generation unit. The generator 13 and the distance calculator 14 are included. Here, the corresponding point search unit 9 has a calibration deviation re-correction function described later. That is, the corresponding point search unit 9 includes a calibration deviation recorrection unit 15.

  The overall flow of the three-dimensional measurement process using the distance measuring apparatus 1 is shown in the flowchart of FIG. FIG. 3 is an explanatory diagram of stereo parallelization processing by the stereo parallelization unit 8 and corresponding point search processing by the corresponding point search unit 9. Prior to explaining the overall flow of the three-dimensional measurement according to FIG. 2, the stereo parallelization process and the corresponding point search process will be explained according to FIG.

  After the stereo camera 2 is placed at an appropriate position relative to the measurement object 5 and the focal length, exposure, and the like are adjusted, image capture between the left camera 6 and the right camera 7 is performed under the control of the CPU 12. Thus, a left camera capture image and a right camera capture image are obtained as shown in FIG.

  The captured image of the stereo camera 2 has an image including distortion, inclination, and displacement as shown in FIG. 3A because of lens distortion, left and right camera angle deviation, and the like. In order to correct this and facilitate the search for corresponding points in the left and right images, a work called camera calibration is performed in advance by the calibration calculation unit 10. Details of the camera calibration will be described later.

  Stereo parallelization processing is performed on the left camera capture image and the right camera capture image by the stereo parallelization unit 8 based on the calibration parameters calculated by the camera calibration and stored in the calibration parameter storage unit 11. Is given. As shown in FIG. 3B, the stereo-parallelized image of the left and right cameras is considered to have pixels corresponding to each other aligned in the vertical direction of the image. That is, for a certain pixel in the stereo-parallelized image of the left camera 6, a corresponding pixel in the stereo-parallelized image of the right camera 7 is always present in the same row. Therefore, if a pixel on the epipolar line is searched, the correspondence between the left and right images can be found.

  The actual corresponding point searching process performed by the corresponding point searching unit 9 is as shown in FIG. First, a square-shaped region 16 called a “correlation window” having p pixels in the vertical and horizontal directions around the pixel of interest is set in the left camera stereo parallelized image. Then, a correlation window 17 having the same size is set in the same row as the correlation window 16 in the right camera stereo parallelized image, and the correlation window 17 is included in the left and right correlation windows 16 and 17 while sliding in the horizontal direction. An evaluation amount indicating the degree of correlation between pixels to be calculated is calculated. In this case, as the evaluation amount indicating the degree of correlation, an amount that can be calculated by pixel calculation such as SAD (Sum of Absolute Differrence) is used.

  When searching from end to end of the predetermined correlation search range, the center pixel of the correlation window 17 in the image after the parallelization of the right camera stereo in the state where the total value of the correlation degree regarding the correlation window is the highest is the left camera stereo. It becomes a corresponding point (corresponding pixel) for the target pixel in the parallelized image. At that time, the distance between the target pixel of the left camera stereo parallelized image and the corresponding pixel of the right camera stereo parallelized image (the distance between the center point of the correlation window 16 and the corresponding point of the correlation window 17) is set in the horizontal direction. Is expressed as a parallax d.

  By performing such a search process for all the pixels of the image after parallelization of the left camera, the value of the parallax d is uniquely determined for each pixel. When the value of the parallax d is expressed as a grayscale image in the image after parallelization of the left camera, a parallax image as the parallax information as shown in FIG.

  If the correlation window 17 in which the degree of correlation is equal to or greater than a predetermined value is not obtained even when searching from end to end of the correlation search range by the corresponding point search process, it is considered that there is no corresponding point. When searching for a portion (occlusion area) that is visible from the left camera 6 but not from the right camera 7 or when the contrast of the measurement object 5 is low, a search result that no corresponding point exists is obtained. There are many. In FIG. 3D, a part of the measurement object 5 is an occlusion area.

  Hereinafter, the three-dimensional measurement processing operation including the stereo parallelization processing and the corresponding point search processing executed under the control of the CPU 12 will be described according to the flowchart shown in FIG.

  In steps S1a and S1b, the left camera 6 and the right camera 7 in the stereo camera 2 capture images at the same timing, and the obtained left camera capture image and right camera capture image are sent to the stereo collimator 8. The Similarly, in the case of the above-described camera calibration, a calibration board, which will be described in detail later, is photographed by the left camera 6 and the right camera 7, and a calibration board photographed image is sent to the calibration calculation unit 10. In step S2, the calibration calculation unit 10 performs a camera calibration process as will be described in detail later. The calculated calibration parameter is stored in the calibration parameter storage unit 11.

  In step S3, the stereo parallelization unit 8 performs the stereo parallelization process as described above on the left camera capture image and the right camera capture image based on the calibration parameters stored in the calibration parameter storage unit 11. Is given. Then, the obtained stereo parallelized images of the left camera 6 and the right camera 7 are sent to the corresponding point search unit 9.

  In step S4, the corresponding point search unit 9 performs the corresponding point search process described above. First, prefiltering is performed in step S4a. Here, the pre-filtering process is an edge enhancement process for absorbing a luminance difference between the stereo-parallelized images of the left and right cameras, a local luminance level variation in the image, and the like. Thereafter, in step S4b, the corresponding point search process is actually performed. In the distance measuring apparatus 1, the left camera 6 is used as a reference, and corresponding points are searched from the stereo-parallelized image of the right camera 7 for all the pixels on the parallelized image of the left camera 6, and FIG. ) Is generated. If no corresponding point is found by this search or if the corresponding point is detected by mistake, noise may be mixed in a part of the parallax image. In particular, when the size of the correlation windows 16 and 17 when searching for corresponding points is relatively small, the ratio of noise tends to increase instead of increasing measurement accuracy. In order to reduce such noise components, post-filtering is performed on the parallax image after the corresponding point search to perform image shaping in step S4c. Here, as the post filter processing, for example, smoothing filter processing such as Gaussian filter processing and median filter processing, or morphological processing such as expansion and contraction is used.

  The parallax image after image shaping obtained in this way is sent to the distance image generation unit 13.

  The processing by the stereo parallelizing unit 8 and the corresponding point searching unit 9 is executed at an appropriate timing based on the control signal sent from the CPU 12.

  In step S5, the distance image generation unit 13 generates a distance image from the parallax image.

The parallax value d of any pixel on the parallax image can be converted into a distance Z in the depth direction from the origin O, which is the center of the optical axis of the image sensor in the left camera 6, by the principle of triangulation. Specifically, the base line length of the stereo camera 2 is A, and the focal length of the left camera 6 is f.
Z = A · f / d
Can be calculated with Therefore, the distance image generation unit 13 determines the parallax d of each pixel from the grayscale image of each pixel of the parallax image, and generates the distance image from the parallax image by determining the distance Z from the parallax value d for each pixel. It can be done.

  In step S6, the distance image generated in step S5 is displayed on the display unit 4 based on the control of the CPU 12. After that, the three-dimensional measurement processing operation is finished.

  Further, in the distance measuring apparatus 1, when one or two points on the distance image are designated and selected by an input means (not shown), if one point is designated, the coordinates of one point are designated. When two values are specified, the distance between the two points is calculated by the distance calculation unit 14 based on the control of the CPU 12, and the calculation result is displayed on the display unit 4.

  As described above, in the distance measuring apparatus 1, the distance image is generated and displayed from the left and right camera images of the measurement object 5 photographed by the stereo camera 2, and any one on the object in the distance image is displayed. The coordinate value of the point and the distance between any two points on the object can be displayed.

  Not only the distance Z in the depth direction but also the distance X in the horizontal direction and the distance Y in the vertical direction can be calculated from the parallax d using the principle of triangulation. In this way, the three-dimensional coordinates (X, Y, Z) of the measurement object 5 with respect to the origin O can be obtained. Furthermore, by multiplying these three coordinate values by an appropriate viewpoint conversion matrix, an image in which the measurement object 5 is viewed from an arbitrary viewpoint can be generated.

  Next, camera calibration for obtaining calibration parameters used when the stereo parallelization unit 8 performs stereo parallelization processing on the left camera capture image and the right camera capture image will be described.

  The left and right camera capture images captured by the stereo camera 2 are images including distortion, inclination, and displacement due to various error factors. Specific error factors can be classified into single factors, that is, error factors of a single camera, and relative factors, that is, factors relating to the positional relationship between the left and right cameras 6 and 7. The single factors include distortion in the radial direction and circumferential direction of the lens, center deviation of the image sensor, and the like. The relative factor is represented by a rotation matrix and a parallel progression between the left and right cameras 6 and 7. These cannot be measured directly.

  Therefore, in the camera calibration, a flat plate on which a pattern having a known geometric shape and dimensions, which is called the calibration board, is prepared and placed in front of the stereo camera 2 at various distances and angles. Shoot as many times as possible. As the calibration board, for example, a checker pattern as shown in FIG. 4A or a grid-like dot (polka dot) pattern as shown in FIG. 4B is used. However, it is not limited to these patterns.

  From the photographed calibration board image, a characteristic point such as a corner point in the case of a checker pattern or a center point of a grid in the case of a grid pattern is found. Next, in the set of left and right camera images, the corresponding feature point is searched. The set of feature point coordinates on the left and right camera images obtained in this way, the condition that two points with known relative distances on the calibration board are associated with the two points on the captured image, and the calibration A plurality of equations can be established based on the condition that one point on the motion board is the same as each point on the left and right camera images. For example, in the case where the checker pattern shown in FIG. 4A is used as the calibration board, if the number of corners of the checker is 12 × 8 and the number of shootings is 10, 12 × 8 × 10 = 960 equations are obtained. . By solving these equations as simultaneous equations, calibration parameters that are unknown numbers can be obtained.

  A specific procedure of such calibration is described in, for example, non-patent document “Z. Zhang,“ IEEE Transactions on Pattern Analysis and Machine Intelligence ”, # 22 (2000), 1330-1334”.

  In actual calibration, as described above, since many error factors (calibration parameters) are obtained as unknowns, the obtained solutions include variations. That is, even if calibration is performed under exactly the same conditions, the obtained calibration parameters are not completely the same. In order to reduce the variation of calibration parameters as much as possible, even if the number of corners and the number of shots of the calibration board are increased and the optimal solution is obtained by the least square method from a large number of equations, it is constant. Variations in quantity remain. Then, the difference between the value of the calibration parameter obtained by calculation and the true value becomes a calibration error, and a certain measurement error is generated. In particular, when the focusing performance of the stereo camera is low, blurring remains when the calibration board is photographed, and there is an error in associating the feature points, which tends to increase the calibration error. It has been known.

  In addition to the fixed amount of calibration error (that is, stationary) as described above, a non-stationary calibration error occurs due to a change with time after calibration. Such unsteady errors include a component that reversibly fluctuates according to the environment, such as a lens shape change due to thermal expansion and contraction, and an irreversible component that occurs due to a mechanical impact or the like.

  The calibration error in actual measurement is the sum of the stationary error component and the unsteady error component including reversible / irreversible due to the change with time after calibration. For this reason, if the stereo camera 2 is made to have a higher pixel and a narrower pixel pitch in order to increase the measurement accuracy, the accuracy required for the calibration will increase accordingly, so both the steady error and the unsteady error are made extremely small. It is necessary to suppress it. This is a major obstacle when performing highly accurate measurement.

  When various calibration parameters include errors, the effects of these calibration parameters on the stereo-parallelized images of the left camera 6 and the right camera 7 are “shift in the vertical direction” and “ It is broadly divided into “deviation” and “other effects”.

The above-mentioned “vertical shift” mainly includes vertical shift (Δc _Y ) of the image sensor of the left camera 6 or right camera 7, vertical shift (ΔY) of the left and right cameras 6, 7, and left and right This is related to the angle shift (θ _X ) around the horizontal axis of the cameras 6 and 7. On the other hand, the “horizontal shift” is mainly the horizontal shift (Δc _X ) of the image sensor, the horizontal shift (ΔX) of the left and right cameras 6 and 7, and the left and right cameras 6 and 7. This is related to the angle deviation (θ _Y ) around the vertical axis. Further, as the “other effects”, the distortion coefficient in the radial direction and the circumferential direction of the lens has a higher-order effect (an effect that differs depending on the center and the periphery of the image, and on the upper, lower, left, and right sides).

  When the stereo camera 2 is made to have a high pixel and a narrow pixel pitch in order to increase the measurement accuracy, among the above three effects on the stereo-parallelized image, “vertical shift” may be a serious problem. . The reason will be described below.

When the stereo-parallelized image is shifted in the left-right direction, the parallax becomes a value different from the true value, resulting in a measurement error. It is assumed that when the parallax deviates from the true value d by Δd, a measurement error in which the depth direction distance measurement value Z is ΔZ occurs. At this time, from the relational expression “Z = A · f / d” of triangulation,
ΔZ / Z = −Δd / (d + Δd) ≈−Δd / d
Is obtained. From this equation, it can be seen that, under the condition of d >> Δd, the ratio Δd / d of the parallax deviation in the left-right direction and the measurement error ΔZ / Z are substantially in the same order.

  For example, if the amount of deviation in the left-right direction is 3%, the measurement error caused thereby is also about 3%. Conversely, if a measurement error of 3% can be tolerated, a deviation amount in the left-right direction is allowed up to 3%. As described above, the deviation in the left-right direction directly affects the measurement error, but even if an error occurs, the measurement itself can be executed without any problem.

  On the other hand, if the stereo-parallelized images of the left and right cameras are shifted in the vertical direction instead of the horizontal direction, there is a possibility that the possibility of the measurement itself is affected rather than the measurement error. For example, when the corresponding point search is performed with the size of the correlation window 16 in the left camera stereo parallelized image set to 13 pixels × 13 pixels and there is no vertical pixel shift, as shown in FIG. In addition, since the corresponding points of the left and right camera stereo parallelized images are perfectly aligned in the vertical direction, the same line of the right camera stereo parallelized image is searched for the left camera stereo parallelized image. However, when the pixel shift amount in the vertical direction is 13 pixels or more, as shown in FIG. 5B, a completely different line of the right camera stereo parallelized image is displayed with respect to the left camera stereo parallelized image. I will search.

  The corresponding point search unit 9 performs the corresponding point search on the assumption that the corresponding points of the left and right camera stereo parallelized images obtained by the stereo parallelization processing in the previous stage are perfectly aligned in the vertical direction. For this reason, in the state shown in FIG. 5 (b), no corresponding point is found and measurement becomes impossible.

  Such a phenomenon is not a problem when the left camera 6 and the right camera 7 are cameras having a relatively wide pixel pitch and a small number of pixels (about tens of thousands to one million pixels). However, when a camera with a small pixel pitch (3 million pixels or more) is used in order to improve the measurement accuracy, it appears remarkably.

  For example, when the size of the image sensor in the left and right cameras 6 and 7 is 1/4 inch and the number of pixels is 5 million pixels, the pixel pitch is about 1.4 μm. Therefore, in order to suppress the vertical pixel shift amount to 13 pixels or less, it is necessary to suppress vertical shift due to calibration to an accuracy of 18.2 μm or less. This is a level that is also affected by dimensional changes due to thermal expansion and contraction after calibration. Therefore, even if all of the above-described parameters that cause the vertical shift are obtained with high accuracy, the vertical shift amount easily exceeds the allowable range due to a calibration error or a temperature change after calibration, etc. Measurement becomes impossible. Thus, even if a camera with a high pixel count is adopted as the left and right cameras 6 and 7 to improve the measurement accuracy, there arises a problem that the measurement accuracy cannot be increased due to the vertical shift of the calibration. Become.

  Therefore, in the distance measuring apparatus 1, in order to solve such a problem, as described in detail below, a calibration parameter re-correction unit 15 of the corresponding point search unit 9 sets search parameters when searching for corresponding points. Thus, calibration deviation re-correction is performed using the obtained search parameters. The procedure of the corresponding point search process including the calibration deviation re-correction process is shown in the flowchart of FIG. FIG. 7 shows the relationship between the vertical shift adjustment amount and the recognition rate during the calibration shift recorrection process. Prior to explaining the procedure of the calibration deviation re-correction processing according to FIG. 6, the calibration deviation re-correction processing will be explained according to FIG.

  In the present embodiment, “recognition rate” is used as the search parameter. That is, prior to searching for corresponding points (corresponding pixels) in the left-right camera stereo parallelized image, the corresponding point search unit 9 including the calibration deviation re-correction unit 15 first pays attention to the left camera stereo parallelized image. The ratio (%) of pixels in which corresponding points in all the pixels in the pixel area to be found are found is calculated as the recognition rate.

  Hereinafter, in detail, the “pixel region of interest” is a pixel region for which the recognition rate is calculated in the image after parallelization of the left camera stereo, and may be the entire image after parallelization of the left camera stereo. It may be a region where the measurement object exists in the image after parallelization of the left camera stereo, or an area in which an error is likely to occur in the subsequent search for corresponding points from the image after parallelization of the left camera stereo is excluded. It may be an area. As described above with reference to FIG. 3C, the correlation windows 16 and 17 are set in the left and right camera stereo parallelized images for all the pixels in the “target pixel area” set in this way. The total value of the degree of correlation between the pixels included in the correlation windows 16 and 17 is calculated while sliding 17 and a corresponding point search is performed with the central pixel of the correlation window 17 in the state where the value is the highest as the corresponding point. . Then, the ratio (%) of the pixels in which the corresponding points in all the pixels in the “target pixel area” are found is calculated as the recognition rate.

  Next, the right camera stereo parallelized image is shifted upward by one pixel with respect to the left camera stereo parallelized image, and corresponding point search and recognition rate calculation are performed in the same manner as described above. Such an operation is performed in a predetermined vertical shift pixel range (for example, from “start position−5 pixel” to “start position + 5 pixel”), and as shown in FIG. 7, the above recognition for the vertical pixel shift amount is performed. A curve showing the rate is obtained. And when the said recognition rate in the peak position of the obtained curve is more than the predetermined value defined beforehand, this peak position is made into the optimal position. Then, the right camera stereo parallelized image is shifted in the vertical direction by the pixel shift amount of the optimal position, and the original correspondence is applied to the pixels on the same line as the correlation window 16 set in the left camera stereo parallelized image. A point search process is performed.

  FIGS. 8A to 8C show changes in parallax images obtained when the pixel shift amount in the downward direction of the right camera stereo parallelized image with respect to the left camera stereo parallelized image is changed. Show. Here, FIG. 8A is a parallax image when the right camera stereo parallelized image is shifted downward by two pixels from the optimal position. FIG. 8B is a parallax image at the optimum position. FIG. 8C shows a parallax image when the pixel is shifted upward by two pixels from the optimum position.

  At the optimum position shown in FIG. 8 (b), corresponding points are searched over substantially the entire screen, and the parallax is calculated. However, at the position shifted by two pixels in the downward direction shown in FIG. 8 (a), there is an area where the search for the corresponding point is impossible in the lower vicinity of the screen, and two pixels in the upward direction shown in FIG. 8 (c). At the shifted position, an area where the corresponding point cannot be searched is generated near the top of the image. For this reason, the recognition rate is lowered. Accordingly, it can be seen that the optimum position can be found as shown in FIG. 8B by obtaining the peak of the recognition rate while shifting the pixels in the vertical direction.

  Hereinafter, the corresponding point search process executed by the corresponding point search unit 9 in step S4b of the three-dimensional measurement processing operation executed under the control of the CPU 12 will be described in detail with reference to the flowchart shown in FIG. When the prefiltering process is completed in step S4a of the three-dimensional measurement processing operation, the corresponding point search processing operation including the calibration deviation recorrection processing starts.

  First, calibration deviation recorrection processing is performed by the calibration deviation recorrection unit 15 of the corresponding point search unit 9.

  In step S4b1, the above-mentioned “target pixel region (hereinafter referred to as the target pixel region)” is set in the left camera stereo parallelized image that is the reference image, and all the pixels in this target pixel region are set as described above. The corresponding point search is performed on the image after the parallelization of the right camera stereo, which is the comparative image.

  In step S4b2, the ratio (%) of pixels in which the corresponding point (corresponding pixel) is found in step S4b1 occupying in all the pixels of the “focused pixel region” is the recognition of the right camera stereo parallelized image. Calculated as a rate.

  In step S4b3, it is determined whether or not the search within the search range, which is the vertically shifted pixel range, is completed. As a result, if completed, the process proceeds to step S4b5. If not completed, the process proceeds to step S4b4. In step S4b4, the right camera stereo parallelized image is moved upward or downward by one pixel. After that, the process returns to step S4b1 to search for the corresponding point between the left camera stereo parallelized image and the right camera stereo parallelized image moved by one pixel in the vertical direction.

  In step S4b5, the vertical shift position of the right camera stereo parallelized image that exhibits the maximum recognition rate among the recognition rates obtained as a result of the corresponding point search within the search range is calculated. In step S4b6, it is determined whether or not the maximum recognition rate obtained in step S4b5 is equal to or higher than a preset setting value (for example, 80%). As a result, if it is not less than the set value, the process proceeds to step S4b7, and if not, the process proceeds to step S4b9.

  In step S4b7, the position of the right camera stereo parallelized image exhibiting the maximum recognition rate calculated in step S4b5 is stored in the calibration parameter storage unit 11 together with the maximum recognition rate as the optimum position.

  In step S4b8, the right camera stereo parallelized image is moved to the optimum position stored in step S4b7. In this way, the calibration deviation recorrection processing by the calibration deviation recorrection unit 15 ends. Thereafter, the above-described original corresponding point search process is performed. That is, as described above with reference to FIG. 3C, the correlation windows 16 and 17 are set in the left and right camera stereo parallelized images, and the correlation windows 17 are slid while the pixels included in the correlation windows 16 and 17 are slid. The total value of the degree of correlation is calculated, and the center pixel of the correlation window 17 in the state where the value is the highest is taken as the corresponding point. Then, a parallax d is obtained in which the distance between the target pixel of the left camera stereo parallelized image and the corresponding point (corresponding pixel) of the right camera stereo parallelized image is expressed in units of pixels in the horizontal direction. Such a corresponding point search process is performed on all the pixels of the image after parallelization of the left camera, and the value of the parallax d of each pixel is obtained. Based on the value of the parallax d, the value shown in FIG. Such a parallax image is generated.

  After that, the corresponding point search processing operation is ended, and the process returns to the post filter processing in step S4c in the three-dimensional measurement processing operation.

  In step S4b9, since it was determined that the maximum recognition rate equal to or higher than the set value was not obtained in step S4b6, the degree of calibration deviation caused by the vertical deviation of the left and right camera stereo parallelized images is determined in this embodiment. It is determined that the degree of correction cannot be corrected by the calibration deviation re-correction process in the embodiment, and the subsequent generation of the parallax image is stopped.

  In step S4b10, since the degree of calibration deviation cannot be corrected by the calibration deviation re-correction processing, a request for camera calibration by the calibration calculation unit 10 is displayed on the display unit 4 based on the control of the CPU 12. The After that, the process returns to the subsequent stage of step S6 in the three-dimensional measurement processing operation, and the three-dimensional measurement processing operation ends.

  As described above, in this embodiment, left and right camera stereo that cannot be corrected by the calibration, that is, cannot be corrected by stereo parallelization processing using the calibration parameter obtained by the camera calibration. The amount of deviation between the parallelized images can be obtained by the calibration deviation recorrection processing by the calibration deviation recorrection unit 15 during the corresponding point search processing by the corresponding point search unit 9 including the calibration deviation recorrection unit 15. Recorrection is performed using data (search parameter: recognition rate). Therefore, the calibration deviation can be re-corrected without increasing the circuit scale of the internal hardware and the calculation time of the distance measuring apparatus 1, thereby improving the accuracy of the three-dimensional measurement.

  Further, in order to re-correct the vertical shift of the left and right camera stereo parallelized images by shifting the right camera stereo parallelized image up and down, particularly a stereo camera having an image sensor with a high pixel count and a narrow pixel pitch Thus, it is possible to reduce the vertical calibration deviation, which is likely to be a problem in the measurement in the above. At that time, one of the left and right camera stereo parallelized image pair (comparison image) is sequentially shifted up and down with respect to the other (reference image), the peak of the recognition rate is searched and the optimum position is set By taking this, it is possible to reliably find the optimal position of the right camera stereo parallelized image when searching for the corresponding point.

  In the above embodiment, the optimum position found in the search for the corresponding pixel region and the value of the recognition rate at the optimum position are stored in the calibration parameter storage unit 11 based on the control of the CPU 12. I remember it. Therefore, when the measurement is performed continuously or intermittently without turning off the power of the distance measuring device 1, the process can be started from the state matched with the optimum position. As described above, by storing the optimum vertical position in the distance measuring device 1, the measurement can be continued quickly under the optimum measurement conditions.

  In addition, when the power of the distance measuring device 1 is once turned off and then turned on again, the measurement may be started from the optimum position. However, discontinuous fluctuation caused by a temperature change or the like occurs. Therefore, it is desirable to perform the calibration deviation re-correction process again and shift the pixel vertically in the right camera stereo parallelized image to obtain the optimum position.

  Further, in the above embodiment, apart from the relatively small state change described so far, a large state change has occurred from the state at the time of calibration due to a long-term change over time, a strong mechanical shock, etc. In this case, a calibration shift that cannot be absorbed in the shifted pixel range in the vertical direction occurs. In such a case, the vertical pixel shift amount-recognition rate curve does not have a clear peak as shown in FIG. Therefore, in step S4b6 in the corresponding point search processing operation shown in FIG. 6, when the obtained maximum recognition rate falls below a preset value (for example, 80%), it is determined that there is no optimal position. Based on the control of the CPU 12, a message for prompting the execution of recalibration is displayed on the display unit 4. By doing so, it is possible to cause the user of the distance measuring device 1 to perform calibration at an appropriate timing, and to prevent a significant decrease in measurement accuracy.

  In the above embodiment, the left camera 6 is used as a reference to search for the corresponding pixel for the target pixel of the left camera stereo parallelized image from the right camera stereo parallelized image. However, it is possible to search for the corresponding pixel for the target pixel of the right camera stereo parallelized image from the left camera stereo parallelized image with the right camera 7 as a reference.

  Even when the corresponding point search unit 9 performs measurement continuously or intermittently without turning off the power of the distance measuring device 1, a predetermined time interval is set during the corresponding point search processing operation. By calculating the recognition rate at the optimal position, the recognition rate as the search parameter is monitored. When the recognition rate falls below an optimum value (for example, the maximum recognition rate calculated in step S4b5) exceeding a predetermined value (for example, 10%), the corresponding point search is performed under the control of the CPU 12. It is also possible to suspend the processing operation and transmit a control signal for instructing resetting of the optimum position by shifting the image after parallelizing the right camera stereo in the vertical direction to the calibration deviation recorrection unit 15. It is. By doing so, an increase in measurement error due to a change with time can be minimized.

  Moreover, in the said embodiment, although the said recognition rate was employ | adopted as a search parameter, it is also possible to use another parameter for the search of an optimal position. For example, when the measurement object 5 is a character written on a relatively flat surface extending in the xy direction, since the position change in the depth direction is small, the calculated value of the parallax d is It is expected to be distributed in a narrow range. Therefore, when the calibration deviation increases, the value of the parallax d is distributed over a wide range due to erroneous detection or the like. Therefore, a method of using a variation (standard deviation) in the value of the parallax d as a search parameter and searching for an upper and lower pixel shift amount that minimizes the search parameter can be considered.

  Also in this case, since the data obtained by the corresponding point search unit 9 can be used as it is, the measurement accuracy can be improved without increasing the circuit scale of the internal hardware of the distance measuring device 1 or increasing the calculation time. .

  Moreover, in the said embodiment, although the said corresponding point search part 9 has the structure containing the calibration deviation recorrection part 15, this invention is not limited to this. The calibration deviation re-correction unit 15 can be realized as a hardware or software block different from the corresponding point search unit 9.

-2nd Embodiment This Embodiment is related with the distance measuring device which recognizes the character stamped on the surface of the measurement object with low contrast, such as a tire of a wheel.

  FIG. 9 shows a block diagram of the distance measuring device in this embodiment. In FIG. 9, the same components as those in the first embodiment are denoted by the same reference numerals, and detailed description thereof is omitted.

  The distance measuring device 21 in the present embodiment is a projector for extracting a two-dimensional random pattern projection 22 and engraved characters as character information from the distance measuring device 1 shown in FIG. 1 in the first embodiment. A character recognition unit 24 such as a binarization unit 23 and an OCR (Optical Character Reader) is added.

  The overall flow of character recognition processing using this distance measuring apparatus 1 is shown in the flowchart of FIG. FIG. 11 is an explanatory diagram of a two-dimensional random texture projected onto the measurement object 5 by the projector 22. FIG. 12 shows an explanatory diagram of the luminance change applied to the base surface and the character upper surface by projecting a two-dimensional random texture onto the surface of the measurement object 5. Prior to describing the overall flow of character recognition according to FIG. 10, the projection of the random pattern and the character recognition of the stamped character will be described according to FIGS.

The projector 22 projects a two-dimensional random texture as shown in FIGS. 11A and 11B on the measurement object 5 during character recognition. The two-dimensional texture projected onto the measurement object 5 is desirably a two-dimensionally random pattern in which periodicity is eliminated as much as possible in order to prevent erroneous detection when searching for corresponding points. In addition to the random binary dot pattern as shown in FIG. 11 (a), a pattern including gray scale multi-tone as shown in FIG. 11 (b), a pattern obtained by coloring these random patterns with random colors, etc. Any pattern can be used. When the measurement object 5 is a low-contrast object such as the tire, the corresponding point cannot be detected by the normal corresponding point search in the first embodiment. There is. For example, as shown in FIG. 12A, when a letter “A” stamped on the surface of the tire is read, the surface of the tire is uniformly black, and each of the base surface and the upper surface of the letter “A” Since there is almost no change in luminance on the surface, no corresponding point can be detected. Also, even in a place where a luminance change exists, a corresponding point cannot be uniquely determined in a portion where a uniform luminance change occurs in the left-right direction such as a horizontal edge of a character. As a result, only the vertical (vertical) edge of the character “A” can be detected, and it is very difficult to correctly recognize the stamped character “A” of the tire.

  On the other hand, when a random binary dot pattern, which is a two-dimensional random texture, is projected onto the surface of the tire, as shown in FIG. 12 (b), the ground surface that is the tire surface and the upper surface of the letter “A” In addition, a fine luminance change caused by the texture is applied. Therefore, due to the randomness of the texture, it is possible to recognize these luminance changes in a one-to-one correspondence with the left and right stereo parallelized images. Therefore, the recognition rate can be dramatically increased as compared with the case where no two-dimensional random texture is projected. As a result, stable character recognition is possible even when the texture change on the surface of the measurement object 5 is poor or the contrast is low.

  Hereinafter, according to the flowchart shown in FIG. 10, the character recognition processing operation including the stereo parallelization processing and the corresponding point search processing executed under the control of the CPU 12 will be described. After the projector 22 starts projecting the two-dimensional random texture onto the surface of the measurement object 5, the character recognition processing operation is started.

  In steps S11a and S11b to step S15, image capture and camera calibration processing by the left camera 6 and the right camera 7 of the stereo camera 2 are performed in the same manner as in steps S1a and S1b to step S5 in the first embodiment described above. Corresponding point search processing including stereo parallelization processing, pre-filter processing, calibration deviation re-correction processing and post-filter processing, and distance image generation are performed.

  In step S16, the binarization process is performed by the binarization unit 23 on the distance image generated in step S15 to generate a binarized image. In this binarization process, for example, as a pixel value indicating the brightness of each pixel in the distance image, a distance corresponding to the upper surface of the character “A” is used as a reference distance, and a pixel representing a distance equal to or less than the reference distance In the case of, the first pixel value indicating the brightness of the character area is set. On the other hand, in the case of a pixel representing a distance exceeding the reference distance, the second pixel value is set to be larger or smaller than the first pixel value. Note that when it is difficult to separate characters and the ground surface by simple binarization processing, such as when the tire ground surface is inclined with respect to the stereo camera 2, so-called “adaptive binarization” or the like. More advanced binarization processing can be performed, or preprocessing for image shaping can be added.

  In step S17, character recognition processing such as optical character recognition is performed based on the binarized image generated in step S16. In step S18, the character information obtained as a result of the character recognition process in step S17 is displayed on the display unit 4 under the control of the CPU 12. After that, the character recognition processing operation is terminated.

  As described above, in the present embodiment, when recognizing characters imprinted on the surface of the measurement object 5 having a low contrast such as the tire, the surface of the measurement object 5 is randomly two-dimensionally. A random binary dot pattern that is a texture is projected. Therefore, it is possible to apply luminance changes to the ground surface and the upper surface of the character, which are the tire surfaces, and easily search for corresponding points by associating these luminance changes one-to-one in the left and right stereo-parallelized images. It becomes possible to do. Therefore, it is possible to accurately calculate the “recognition rate” calculated by the ratio (%) of pixels in which the corresponding points in all the pixels of the target pixel area in the left camera stereo parallelized image are found. Thus, the recognition rate in the character recognition can be increased.

  In the second embodiment, the character recognition is described as an example. However, the present invention is not limited to the character recognition, and what is imprinted on the surface of the measurement object 5 having a low contrast is used. It can be a symbol or pattern.

Other Embodiments In the first and second embodiments, two cameras, the left camera 6 and the right camera 7, are mounted, and corresponding points are searched from the left and right image pairs. However, the number of cameras is not limited to two, and the present invention can be applied to a multi-view stereo such as a three-eye or four-eye camera. In the case of an N-eye camera (N ≧ 3), it is desirable to use any one of the cameras as a reference camera and perform calibration and calibration deviation correction for the other (N−1) cameras. That is, (N−1) cameras are calibrated with respect to the reference camera, and (N−1) sets of calibration parameters are stored in the calibration parameter storage unit. When a calibration deviation occurs, each of the (N-1) cameras is slid vertically with respect to the reference camera to perform calibration deviation re-correction to obtain the optimum position.

  In the first and second embodiments, the case where the measurement object 5 is a stationary object has been described for the sake of simplicity. However, even if the measurement object 5 is a moving object, the left camera 6 and the right camera 7 are synchronized to shoot a moving image, and each frame of the moving image is taken out as a still image, whereby the first and second images are taken. Processing similar to that in the embodiment can be performed. However, when the three-dimensional measurement process and the character recognition process are performed from a moving image, it is necessary to perform a corresponding point search and a parallax calculation process within one frame period, and thus a high-speed calculation circuit is required.

  The three-dimensional measuring device, which is each of the distance measuring devices, can be used as an independent measuring device for industrial use, consumer use, and other purposes, or can be incorporated into a part of a general-purpose portable information terminal or the like. In addition, a part or all of the arithmetic circuit can be used as an integrated circuit (IC).

  In addition, various modifications can be made without departing from the spirit of the present invention.

  The stereo three-dimensional measuring apparatus of the present invention is useful when measuring a distance and shape using a plurality of cameras, and can be used as an independent measuring apparatus for industrial, consumer, and other purposes. It can be used by being incorporated into a part of a general-purpose portable information terminal or the like, or a part or all of an arithmetic circuit can be used as an integrated circuit IC.

1,21 ... Distance measuring device,
2 ... Stereo camera,
3 ... measurement calculation part,
4 ... display part,
5 ... Measurement object,
6 ... Left camera,
7 ... Right camera,
8 ... Stereo parallelizing section,
9 ... corresponding point search part,
10: Calibration calculation unit,
11: Calibration parameter storage unit,
12 ... CPU,
13 ... Distance image generation unit,
14 ... distance calculation part,
15 ... Calibration deviation re-correction unit,
16, 17 ... correlation window,
22 ... Projector,
23... Binarization unit,
24: Character recognition unit.

Claims (7)

A plurality of cameras that take images of measurement objects and acquire images;
A calibration calculation unit that calculates a calibration parameter by performing camera calibration using captured images of a plurality of camera calibration boards acquired by the plurality of cameras;
A storage unit for storing the calculated calibration parameter;
A stereo parallelization process is performed to correct an error including at least one of distortion, inclination, and shift included in the images of the plurality of measurement objects acquired by the plurality of cameras using the calibration parameter. Stereo parallelization processing,
A calibration deviation recorrector that recorrects the shift between the plurality of images after the stereo parallelization processing using data obtained by searching for pixels corresponding to each other in the images after the stereo parallelization processing; ,
The pixels corresponding to each other in the images of the plurality of measurement objects recorrected by the calibration deviation recorrection unit are searched as corresponding points, and the distances between the searched corresponding points are the plurality of the plurality of corresponding points. A corresponding point search unit that generates disparity information indicating a distance in a direction in which at least two of the cameras are arranged;
A stereo three-dimensional measurement apparatus comprising: a distance information generation unit that obtains distance information from the parallax information generated by the corresponding point search unit to the measurement object. The stereo three-dimensional measurement apparatus according to claim 1,
When the direction in which the two cameras are arranged is the x direction and the direction orthogonal to the x direction in the plane of the image captured by the two cameras is the y direction,
The calibration deviation re-correction unit re-corrects at least the deviation in the y direction between the images after the stereo parallelization process. In the stereo three-dimensional measuring apparatus according to claim 1 or 2,
A stereo three-dimensional measurement apparatus comprising a projector that projects a two-dimensional random texture onto the measurement object. In the stereo three-dimensional measuring apparatus according to claim 2 or 3,
The calibration deviation re-correction unit sequentially selects image pairs each including a reference image serving as a reference and a remaining comparison image compared with the reference image from the plurality of images after the stereo parallelization processing. A search parameter as the data is obtained by searching for a corresponding pixel in both images while shifting the comparison image of the selected image pair in the y direction with respect to the reference image, and based on the obtained search parameter Determining an optimal shift position of the comparison image, and re-correcting the shift between the plurality of images by shifting the comparison image in the y direction to the optimal shift position with respect to the reference image. Stereo three-dimensional measuring device. The stereo three-dimensional measurement apparatus according to claim 4,
The storage unit also stores the optimal shift position obtained by the calibration shift recorrection unit,
The calibration deviation re-correction unit, when re-correcting the deviation between the plurality of images, uses the optimum shift position stored in the storage unit to compare the comparison image with the reference image. A stereo three-dimensional measuring apparatus characterized by being shifted in the y direction up to the optimum shift position. The stereo three-dimensional measurement apparatus according to claim 5,
The corresponding point search unit monitors the search parameter when performing the corresponding point search and the generation of the disparity information, and when the value of the search parameter deteriorates beyond a predetermined value beyond a predetermined value. A stereo three-dimensional measurement apparatus characterized by instructing the calibration shift re-correction unit to reacquire the optimal shift position. The stereo three-dimensional measurement apparatus according to claim 6,
A display unit that displays at least the distance information generated by the distance information generation unit;
The calibration deviation re-correction unit, when the obtained values of the plurality of search parameters are less than a preset setting value, information that prompts the calibration calculation unit to execute the camera calibration A stereo three-dimensional measuring apparatus characterized by being displayed on a display unit.

Images