JP2011247759A - Three-dimensional shape measuring device, calibration method and robot - Google Patents

Abstract

To perform calibration in the height direction in three-dimensional shape measurement efficiently and with high accuracy.
A calibration block having a staircase shape, a mounting table driving unit for moving a mounting table on which the staircase is mounted, a light cutting line detection unit for detecting a light cutting line from a captured image, and a light cutting. A reflection position that detects a feature point from a line, calculates a feature point coordinate value, and generates feature point information that associates the feature point coordinate value, the movement pitch of the mounting table 40, and predetermined attribute data of the calibration block Based on the calculation unit 23, a conversion matrix calculation unit 24 for calculating a conversion matrix for converting the feature point coordinate value and the movement pitch into a three-dimensional coordinate value of the feature point in the world coordinate system, and the feature point information and the conversion matrix And a three-dimensional coordinate conversion unit 25 that generates a calibration data by obtaining a combination of the two-dimensional coordinate value of the image plane of the captured image and the corresponding three-dimensional coordinate value of the world coordinate system.
[Selection] Figure 2

Description

  The present invention relates to a three-dimensional shape measuring apparatus, a calibration method, and a robot.

Light that irradiates the measurement target object with the slit light emitted from the slit light source, captures the bright line (light cutting line) of the slit light reflected on the surface of the measurement target object, and measures the three-dimensional shape of the measurement target object In the three-dimensional shape measuring apparatus using the cutting method, calibration (calibration) is required for associating the position in the image of the measurement target object imaged by the camera with the position in the real space.
Therefore, conventionally, a calibration device is known that can simultaneously perform camera distortion correction and height direction calibration to reduce the time required for the entire calibration work (see, for example, Patent Document 1). In the calibration apparatus disclosed in Patent Document 1, a stair-like block is arranged in each row, and calibration is performed using a calibration block in which every other stair-like block is arranged one step lower than the previous row. .

JP 2007-33039 A

  However, since the calibration apparatus disclosed in Patent Document 1 obtains coordinates on an image from a lattice figure obtained by synthesizing bright lines on a calibration block and uses them for calibration, the entire calibration block is reliably measured. Therefore, it was necessary to correspond to the lattice figure on the image, and the calibration work took a long time.

  Therefore, the present invention has been made in view of the above problems, and provides a three-dimensional shape measuring apparatus and a calibration method that efficiently and accurately perform calibration in the height direction in three-dimensional shape measurement. With the goal.

[1] In order to solve the above-described problem, a three-dimensional shape measuring apparatus according to an aspect of the present invention includes a light source unit that emits slit light, a mounting table on which a calibration block is mounted, and the calibration An imaging unit that captures an optical cutting line of the slit light irradiated to the block for use, a moving unit that moves at least one of the combination of the mounting table, the light source unit, and the imaging unit, and the imaging unit A light cutting line detection unit for detecting the light cutting line from the captured image; a feature point is detected from the light cutting line to calculate a feature point coordinate value; the feature point coordinate value, a movement pitch of the moving unit, and the A feature point information generating unit that generates feature point information associated with predetermined attribute data of a calibration block; and the feature points in the world coordinate system from the feature point coordinate values and the movement pitch Based on the transformation matrix calculation unit for calculating a transformation matrix to be transformed into a three-dimensional coordinate value, the feature point information and the transformation matrix, the two-dimensional coordinate value of the image plane of the captured image and the two-dimensional coordinate value A three-dimensional coordinate conversion unit that obtains a combination with the three-dimensional coordinate values of the world coordinate system and generates calibration data.
According to one aspect of the present invention, the three-dimensional shape measurement apparatus uses the feature point coordinate value of the feature point obtained by imaging the calibration block and the movement pitch of the moving unit to obtain the third order of the feature point in the world coordinate system. Since the transformation matrix to be converted into the original coordinate values is obtained and the calibration data is generated, there is no need to measure the entire calibration block reliably as in the conventional case, and the calibration in the height direction is efficiently and highly accurate. It can be carried out.

[2] The three-dimensional shape measuring apparatus according to [1], wherein the light section line detection unit detects the light section line by obtaining a barycentric position of luminance from the captured image.
Thereby, the three-dimensional shape measuring apparatus can detect the light cutting line with high accuracy by removing the influence of noise included in the light cutting line of the captured image.

[3] In the three-dimensional shape measuring apparatus according to [1] or [2], the calibration block has a plurality of plane portions having different height dimensions from the bottom portion placed on the mounting table. Each of the plurality of plane portions has a linear outline that is non-parallel to the light cutting line generated by the irradiation of the slit light, and a pattern having different light reflectivity is formed with the outline as a boundary. And
Here, it is preferable that geometric patterns having different light reflectivities are formed on each of the plurality of plane portions of the calibration block. As a result, the three-dimensional shape measuring apparatus can detect feature points at various positions from the same height plane when the slit light is irradiated onto the pattern of the plane portion.

[4] In the three-dimensional shape measurement apparatus according to [3], the feature point information generation unit calculates a vertex coordinate value of the pattern based on a plurality of the feature point coordinate values, and the vertex coordinate value and the Vertex information associating the moving pitch of the moving unit with predetermined attribute data of the calibration block is generated and added to the feature point information.
Thereby, the three-dimensional shape measuring apparatus can obtain the feature point information finer than the moving pitch of the moving unit and without being affected by the pixel pitch of the captured image.

[5] In order to solve the above-described problem, a calibration method for a three-dimensional shape measurement apparatus according to an aspect of the present invention is a mounting method in which a calibration block is placed in the calibration method for a three-dimensional shape measurement apparatus. Moving at least one of a combination of a mounting table, a light source unit that emits slit light, and an image pickup unit that picks up an optical cutting line of the slit light irradiated to the calibration block; and an image picked up by the image pickup unit Detecting the light cutting line from the image; detecting a feature point from the light cutting line; calculating a feature point coordinate value; and calculating the feature point coordinate value, the moving pitch of the moving unit, and the calibration block Generating feature point information in association with predetermined attribute data, and calculating the world position from the feature point coordinate value and the movement pitch. A step of calculating a transformation matrix to be transformed into a three-dimensional coordinate value of the feature point in the system, a two-dimensional coordinate value of the image plane of the captured image and the two-dimensional coordinate value based on the feature point information and the transformation matrix Generating a calibration data by obtaining a combination with a three-dimensional coordinate value of the world coordinate system corresponding to.
According to one aspect of the present invention, the three-dimensional shape measurement apparatus uses the feature point coordinate value of the feature point obtained by imaging the calibration block and the movement pitch of the moving unit to obtain the third order of the feature point in the world coordinate system. Since the transformation matrix to be converted into the original coordinate values is obtained and the calibration data is generated, there is no need to measure the entire calibration block reliably as in the conventional case, and the calibration in the height direction is efficiently and highly accurate. It can be carried out.

[6] In order to solve the above problems, a robot according to an aspect of the present invention is directed to a light source unit that emits slit light, a mounting table on which a calibration block is mounted, and irradiation to the calibration block. An imaging unit that captures an optical cutting line of the slit light, a hand unit to which any one of the mounting table and the combination of the light source unit and the imaging unit is attached, and the hand unit is movably attached A feature point is detected by calculating a feature point coordinate value by detecting a feature point from the optical cutting line, and a feature point coordinate value is calculated from the optical cutting line. A feature point information generating unit that generates feature point information in which a point coordinate value, a movement pitch in one direction of the hand unit, and predetermined attribute data of the calibration block are associated; and the feature point locus An image of a captured image based on the transformation matrix calculation unit that calculates a transformation matrix that transforms the value and the moving pitch into a three-dimensional coordinate value of the feature point in the world coordinate system; and the feature point information and the transformation matrix And a three-dimensional coordinate conversion unit that generates a calibration data by obtaining a combination of a two-dimensional coordinate value of a plane and a three-dimensional coordinate value of the world coordinate system corresponding to the two-dimensional coordinate value.
According to one aspect of the present invention, the robot can obtain the three-dimensional feature point in the world coordinate system from the feature point coordinate value of the feature point obtained by imaging the calibration block and the unidirectional movement pitch of the hand unit. Since a conversion matrix to be converted into coordinate values is obtained and calibration data is generated, there is no need to reliably measure the entire calibration block as in the past, and calibration in the height direction is performed efficiently and with high accuracy. be able to.

1 is a perspective view of a three-dimensional shape measuring apparatus and a calibration block schematically showing how a three-dimensional shape measuring apparatus according to an embodiment of the present invention performs calibration. It is a block diagram which shows the main function structures of the three-dimensional shape measuring apparatus in the embodiment. It is a perspective view of the appearance of a calibration block in the same embodiment. In the embodiment, it is an example of a pattern that is patterned on each upper plane of a calibration block. 5 is a flowchart illustrating a procedure of calibration data generation processing executed by the three-dimensional shape measurement apparatus in the embodiment. In the same embodiment, it is the figure which showed typically a mode that a pattern passed a bright line when the block for a calibration was conveyed. 4 is a flowchart illustrating a processing procedure of 3D shape measurement executed by the 3D shape measurement apparatus in the embodiment. FIG. 3 is a perspective view of a robot, a calibration block, and a mounting table, showing a state where a robot incorporating a part of the functions of the three-dimensional shape measuring apparatus is performing calibration.

Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings. FIG. 1 is a perspective view of a three-dimensional shape measuring apparatus and a calibration block schematically showing how a three-dimensional shape measuring apparatus according to an embodiment of the present invention performs calibration. A three-dimensional shape measurement apparatus 1 in the figure is an apparatus that measures the external shape of a measurement target object (including a calibration block) by a light cutting method, and operates by switching to a calibration mode or a normal measurement mode. . The light cutting method refers to, for example, a bright line (light cutting) in which the slit light source irradiates the measurement target object with slit light while the slit light source and the measurement target object are moved relative to each other, and the imaging device reflects on the surface of the measurement target object. Line) to acquire a two-dimensional bright line image, and the image processing apparatus obtains a three-dimensional shape of the measurement target object based on the bright line image.
The three-dimensional shape measurement apparatus 1 includes a slit light source unit 11, an imaging unit 12, and a mounting table 40. In FIG. 2, the mounting table 40 is mounted with a calibration block 100 used for matching the position in the image captured by the imaging unit 12 with the position in the real space.

  FIG. 2 is a block diagram showing the main functional configuration of the three-dimensional shape measuring apparatus 1. In the figure, the same components as those shown in FIG. As shown in FIG. 2, the three-dimensional shape measurement apparatus 1 includes an optical measurement unit 10, a control unit 20, a mounting table driving unit 30, and a mounting table 40. Hereinafter, the configuration of the three-dimensional shape measuring apparatus 1 will be described with reference to FIGS. 1 and 2 together.

The optical measurement unit 10 is a measurement unit that performs three-dimensional shape measurement by a light cutting method, and includes a slit light source unit 11 and an imaging unit 12.
The slit light source unit 11 is a light source that emits slit light SL. The slit light SL spreads in a fan shape and is emitted into the space. As the slit light SL, for example, laser light is preferably used. The slit light source unit 11 is installed with a predetermined angle at which the central optical axis of the slit light SL is not parallel to the vertical axis.

  The mounting table 40 on which the calibration block 100 is placed is moved in the transport direction A by driving the mounting table driving unit 30, whereby the calibration block 100 is transported in the transport direction A. When the slit light SL emitted from the slit light source unit 11 is irradiated onto the stepped plane provided on the upper surface side of the calibration block 100 being conveyed, the reflected light RL is reflected from the irradiated portion.

  The imaging unit 12 is, for example, a digital camera or a digital video camera. The imaging unit 12 is a position where the optical axis of the optical system is inclined at a predetermined angle that is not parallel to the central optical axis of the slit light SL, specifically, a position where the reflected light RL is incident on the optical system and can be imaged. Installed. At this time, the reflection part of the calibration block 100 is visualized as a bright line LL which is a light cutting line. The imaging unit 12 captures the bright line LL and supplies the captured image data to the control unit 20.

  The control unit 20 controls the entire three-dimensional shape measuring apparatus 1 and includes a CPU (Central Processing Unit) and a semiconductor storage unit (both not shown). The control unit 20 is functionally composed of a measurement control unit 21, a light section line detection unit 22, a reflection position calculation unit (feature point information generation unit) 23, a transformation matrix calculation unit 24, and a three-dimensional coordinate conversion unit 25. And the storage unit 26.

The measurement control unit 21 controls the optical measurement unit 10 and the mounting table driving unit 30. Specifically, the measurement control unit 21 controls the slit light source unit 11 to start and stop the emission of the slit light SL. Further, the measurement control unit 21 controls the imaging unit 12 to start and stop imaging of the bright line LL reflected on the surface of the measurement target object, or to supply the optical cutting line detection unit 22 with the captured image data. Or do. Further, the measurement control unit 21 controls the mounting table driving unit 30 to move the mounting table 40 at least in the transport direction A.
In addition, the measurement control unit 21 supplies the movement pitch of the mounting table driving unit 30 to the reflection position calculation unit 23. This movement pitch is the movement amount of the mounting table 40 per unit time.

The light section line detection unit 22 takes in the captured image data supplied from the image capturing unit 12 and detects the position of the light section line from the image. Depending on the imaging performance of the imaging unit 12, the image of the bright line LL (bright line image) included in the image may include noise or be captured thickly. Therefore, the light section line detection unit 22 detects the position of the light section line with high accuracy by determining the barycentric position of the luminance from the image.
Specifically, the light section line detection unit 22 in the x-axis direction in the window from the coordinate value (i _min , j _min ) to the coordinate value (i _max , j _max ) when the image plane is the xy coordinate system. The luminance gravity center position _xc is obtained by calculating the following equation (1). In equation (1), (x _i , y _j ) is a coordinate value of the image plane, and I (x _i , y _j ) is a luminance value at the coordinate value (x _i , y _j ).

Then, the light section line detecting unit 22 stores a y _c is the position of the y-axis direction corresponding to the position of the gravity center position x _c Toko, barycentric coordinates value in the window (x _{c, y c)} as. Further, the light section line detection unit 22 calculates the expression (1) while shifting the window to detect the position of the light section line.

The reflection position calculation unit 23 detects a feature point that is a change point in luminance of the light section line detected by the light section line detection unit 22 and calculates a feature point coordinate value on the image plane. Then, in the calibration mode, the reflection position calculation unit 23 obtains the feature point coordinate value, the movement pitch supplied from the measurement control unit 21, and the predetermined attribute data of the calibration block 100 corresponding to the feature point. The associated feature point information is generated and stored in the storage unit 26. This feature point information will be described in detail in the operation description section described later.
The transformation matrix calculation unit 24 sets a transformation matrix for obtaining a three-dimensional coordinate value of the world coordinate system that defines the position of the measurement target object from the two-dimensional coordinate value of the image plane and the movement pitch of the mounting table driving unit 30 ( The transformation matrix set) is calculated and stored in the storage unit 26.

  In the calibration mode, the three-dimensional coordinate conversion unit 25 converts the two-dimensional coordinate value of the image plane of the captured image data and the two-dimensional coordinate value based on the feature point information and the conversion matrix set respectively read from the storage unit 26. Find a combination with the corresponding 3D coordinate value of the world coordinate system. Then, the three-dimensional coordinate conversion unit 25 performs calibration for converting the two-dimensional coordinate value to the three-dimensional coordinate value, generates calibration data, and stores it in the storage unit 26. In the normal measurement mode, the three-dimensional coordinate conversion unit 25 calculates the three-dimensional coordinates of the measurement target object based on the result of calibration, and then reads the calibration data stored in the storage unit 26 to perform measurement. The three-dimensional coordinate value is corrected according to the movement pitch of the target object.

The storage unit 26 stores the feature point information generated by the reflection position calculation unit 23, the conversion matrix set calculated by the conversion matrix calculation unit 24, and the calibration data generated by the three-dimensional coordinate conversion unit 25. The storage unit 26 stores in advance reference data including the external dimensions of the calibration block 100 and information related to the pattern position.
The mounting table driving unit 30 moves the mounting table 40 in a two-dimensional direction on a plane parallel to the mounting surface based on the control information of the moving direction instruction and the moving pitch supplied from the measurement control unit 21.

  FIG. 3 is a perspective view of the appearance of the calibration block 100. As shown in the figure, the calibration block 100 is integrally formed with four steps. Note that the number of steps is not necessarily four, but may be a plurality of steps. The calibration block 100 may be hollow or concave. The calibration block 100 is mounted with its bottom 110 aligned with the mounting surface of the mounting table 40, and the bottom 110 has a surface or at least four support portions in order to ensure the stability of installation. Have. A stepped plane 101-104 that is parallel to the placement surface of the bottom 110 and has different vertical dimensions is provided in the stepped shape portion on the opposite side to the bottom 110.

  Since the calibration block 100 is irradiated with, for example, laser light as the slit light SL, the temperature of the irradiated portion tends to increase. Therefore, the calibration block 100 is preferably made of a material having a small coefficient of thermal expansion. The material of the calibration block 100 can be, for example, carbon black, glass, or stainless steel. Among these, carbon black is excellent in that the mass can be reduced as compared with glass and stainless steel, and is preferable as a material.

  A predetermined pattern P is formed (patterned) on each of the upper planes 101 to 104 of the calibration block 100. FIG. 4 is an example of a pattern that is patterned on each of the upper planes 101 to 104 of the calibration block 100. As shown in the figure, the pattern P includes a pattern AP and a pattern RP which are geometric patterns. In the figure, a hatched pattern AP and a plurality of unpatterned patterns RP are represented as being provided in the pattern P, but these pattern colors are limited to the patterns and colors expressed in the figure. It is not something. What is important as the pattern P is that the pattern AP and the pattern RP are patterned so as to have different light reflectivities with respect to the irradiation of the slit light SL. In the present embodiment, the pattern RP is patterned so that the reflectance is higher than that of the pattern AP. The geometric pattern expressed on the pattern P is a unique pattern as a row (a set of four patterns RP).

  As a patterning method of the pattern P on the calibration block 100, it is preferable to use a fine patterning technique such as a thin film deposition method, an etching method, or a photoresist method.

  In the calibration block 100, the external dimensions and the positions of the feature points of the pattern P (vertex of each pattern RP, etc.) are measured by a shape measuring device other than the three-dimensional shape measuring device 1 according to this embodiment. For example, for each pattern RP, reference data in which the vertex coordinate value and the height dimension from the bottom 110 to the stepped plane on which the pattern RP is patterned is associated with the identification number of the pattern RP is created. Yes. This reference data is stored in advance in the storage unit 26 as described above.

  Next, the operation in the calibration mode in which the three-dimensional shape measurement apparatus 1 performs calibration using the calibration block 100 will be described. FIG. 5 is a flowchart showing a procedure of “calibration data generation processing” executed by the three-dimensional shape measuring apparatus 1. After the calibration block 100 is mounted on the mounting table 40 so that the transport direction is the direction of the transport direction A shown in FIG. 3, the three-dimensional shape measuring apparatus 1 performs calibration with respect to the measurement control unit 21. When a mode operation start instruction is given, the processing of the flowchart shown in FIG. 5 is started.

  In step S501, the measurement control unit 21 instructs the slit light source unit 11 to start irradiation. And the slit light source part 11 which received this instruction | indication starts irradiation of the slit light SL. Next, in step S502, the measurement control unit 21 instructs the imaging unit 12 to start imaging. Upon receiving this instruction, the imaging unit 12 starts imaging. Next, in step S503, the measurement control unit 21 instructs the mounting table driving unit 30 to start driving. Upon receiving this instruction, the mounting table drive unit 30 starts moving the mounting table 40.

  Next, in step S <b> 504, the imaging unit 12 captures the bright line LL reflected on the surface of the calibration block 100 and supplies the captured image data to the light section line detection unit 22. Next, in step S505, the light section line detection unit 22 captures captured image data, and determines the position of the center of gravity of the brightness from the image as described above, thereby detecting the position of the light section line from the image. Next, in step S <b> 506, the reflection position calculation unit 23 searches for a feature point from the light section line detected by the light section line detection unit 22. Details of the “feature point search process” will be described later.

  Next, in step S507, when the reflection position calculation unit 23 detects a feature point from the light section line (step S507: YES), the process proceeds to step S508. On the other hand, if the reflection position calculation unit 23 does not detect a feature point from the light section line (step S507: NO), the process proceeds to step S509. In step S508, the reflection position calculation unit 23 calculates the feature point coordinate value on the image plane of the detected feature point. Next, the reflection position calculation unit 23 associates the feature point coordinate value, the movement pitch supplied from the measurement control unit 21, and predetermined attribute data of the calibration block 100 corresponding to the feature point. Information is generated and stored in the storage unit 26. Details of the “feature point information generation process” will be described later.

  Next, in step S509, the measurement control unit 21 determines whether or not the four patterns P patterned on the four upper planes 101 to 104 of the calibration block 100 have been scanned with the bright lines LL. If it is determined that the scanning is completed (S509: YES), the process proceeds to step S510. If it is determined that the scanning is not completed (S509: NO), the process returns to step S504. In step S <b> 510, the reflection position calculation unit 23 reads the feature point information stored in the storage unit 26, interpolates the data, and stores the data in the storage unit 26. Details of this “feature point information interpolation process” will be described later.

  Next, in step S511, the transformation matrix calculation unit 24 obtains a three-dimensional coordinate value of the world coordinate system that defines the position of the measurement target object from the two-dimensional coordinate value of the image plane and the movement pitch of the mounting table driving unit 30. A transformation matrix set is calculated and stored in the storage unit 26. Details of this “transform matrix set calculation process” will be described later.

  Next, in step S512, the three-dimensional coordinate conversion unit 25, based on the feature point information and the transformation matrix set read from the storage unit 26, respectively, the two-dimensional coordinate values of the image plane of the captured image data and the corresponding world coordinates. The combination with the three-dimensional coordinate value of the system is obtained. Then, the three-dimensional coordinate conversion unit 25 performs calibration for converting the two-dimensional coordinate value to the three-dimensional coordinate value, generates calibration data, and stores it in the storage unit 26. Details of the “calibration data generation process” will be described later.

  Next, the “feature point search process” and the “feature point information generation process” by the reflection position calculation unit 23 in the above-described operation description will be specifically described. FIG. 6 shows that when the calibration block 100 is transported in the transport direction A, the pattern P (in the figure, the pattern AP and one pattern RP are enlarged) passes through the bright line LL. It is the figure which showed the mode typically. In the figure, for easy understanding of the drawing, the relative positional relationship when the pattern P is fixed and the bright line LL is moved from the bright line LL1 to the bright line LL6 is shown.

As shown in FIG. 6, the imaging unit 12 sequentially captures the bright lines LL1, LL2, LL3,..., LL6 and supplies them to the light section line detection unit 22. First, in the figure, when the imaging unit 12 captures the bright line LL1 and supplies the captured image data to the optical cutting line detection unit 22, the optical cutting line detection unit 22 calculates the barycentric coordinate position of the luminance from the image. The position of the light cutting line is detected and supplied to the reflection position calculation unit 23.
Next, the reflection position calculation unit 23 detects locations where the luminance changes on the light section line obtained from the bright line LL1 as feature points C1 and C2. That is, the feature points C1 and C2 correspond to the reflectance change points on the pattern P in which the bright line LL1 is reflected. When the reflection position calculation unit 23 detects the feature points C1 and C2, the reflection position calculation unit 23 calculates feature point coordinate values corresponding to the feature points C1 and C2 in the image. Then, feature point information is generated by associating the feature point coordinate values, the movement pitch supplied from the measurement control unit 21, and the predetermined attribute data of the calibration block 100 corresponding to the feature points C1 and C2. Store in the storage unit 26.

  The predetermined attribute data is information including the height dimension from the bottom 110 of the calibration block 100 to the upper plane on which the pattern RP corresponding to the target feature point is patterned, and the identification number of the pattern RP. It is. For example, the pattern RP corresponding to the feature points C1 and C2 in FIG. 6 is a geometric pattern in the pattern P patterned on the stepped plane 103 in FIG. 3, and is the first geometric after the start of scanning. In the case of the 95th geometric pattern counted from the target pattern, the attribute data includes a height dimension from the bottom 110 to the stepped plane 103 and an identification number “95”. The reflection position calculation unit 23 reads the reference data from the storage unit 26, and the bright line LL is from the first line of the pattern RP on the upper plane 104 of the calibration block 100 to the fourth line of the pattern RP on the upper plane 101 (final). The pattern RP can be specified and attribute data can be obtained by counting the number of geometric patterns while scanning up to (row).

  Similar to the operation for detecting the feature points C1 and C2 and the operation for storing the feature point information in the storage unit 26, the reflection position calculation unit 23 includes the feature points C3-C8 illustrated in FIG. A point is detected and feature point information is generated and stored in the storage unit 26.

Next, the “interpolation process of feature point information” by the reflection position calculation unit 23 will be specifically described. In the storage unit 26, feature point information related to the feature points C1-C8 detected by scanning the bright lines LL1-LL6 is already stored. The reflection position calculation unit 23 reads out the feature point information from the storage unit 26, and first calculates the vertex coordinate value of the vertex T1 of the pattern RP from the feature point coordinate values corresponding to the feature points C1-C4. That is, the vertex T1 is a point where a line segment passing through the feature points C1 and C3 and a line segment passing through the feature points C2 and C4 intersect. Similarly, the vertex coordinate value of the vertex T4 of the pattern RP is calculated from the feature point coordinate values corresponding to the feature points C5-C8. Similarly, the vertex coordinate values of the vertices T2 and T3 of the pattern RP are calculated from the feature point coordinate values corresponding to the feature points C1 to C8.
The reflection position calculation unit 23 associates each vertex coordinate value of the vertex T1-T4, the movement pitch supplied from the measurement control unit 21, and predetermined attribute data of the calibration block 100 corresponding to the vertex T1-T4. The generated vertex information is generated and added to the feature point information stored in the storage unit 26. Thereby, the reflection position calculation unit 23 can obtain the feature point information finer than the bright line movement pitch and without being affected by the pixel pitch.

  By setting the pattern RP to a geometrically simple shape that does not have an outline parallel to the bright line LL, the coordinate values of the vertices of the pattern RP can be easily obtained as described above. Moreover, since the vertex is calculated from the geometric arrangement of the detected feature points, the vertex coordinates can be obtained with high accuracy without depending on the resolution of the imaging unit 12 and the moving pitch of the mounting table 40. .

  The above explanation is an example of calculating the vertices T1 to T4 for one geometric pattern of the patterns RP. However, such calculation is applied to all the other patterns RP of the pattern P, every other column or 1 This is done according to a predetermined rule such as every other line.

  Next, the “transformation matrix set calculation process” performed by the transform matrix calculation unit 24 and the “calibration data generation process” performed by the three-dimensional coordinate conversion unit 25 will be specifically described. When the two-dimensional coordinate value of the image plane of the captured image data is (xy) and the three-dimensional coordinate value of the world coordinate system that defines the position of the measurement target object is (XYZ), each homogeneous coordinate value q (Bold body) and Q (bold body) are expressed by Expression (2), and a coordinate conversion expression is expressed by Expression (3). Note that the description of “bold” indicates that the character immediately before it is written in bold, and that the data indicated by the character is a matrix or a vector.

In equation (3), s is a scale parameter. M (bold body) is an internal parameter matrix of the imaging unit 12.
The internal parameter matrix M (bold body) includes the following elements.
f _x : Focal length in the x-axis direction (expression in pixel units)
f _y : focal length in the y-axis direction (expressed in pixel units)
c _x : principal point (x coordinate) that is the center of the image
c _y : principal point (y coordinate) that is the center of the image

W (bold body) is an external parameter matrix of the imaging unit 12 and includes the following elements.
R (bold body): rotation matrix representing transformation from world coordinates to camera coordinates t (bold body): parallel progression representing transformation from world coordinates to camera coordinates The world coordinates are at the beginning of the calibration block 100. It is the coordinate of the position.

When the imaging lens included in the imaging unit 12 has lens distortion, the transformation matrix calculation unit 24 calculates the expression (4) with the two-dimensional coordinate value (xy) of the image plane as (x _d y _d ) and calculates the lens. Correct distortion. In Equation (4), k ₁ , k ₂ , and k ₃ are coefficients indicating lens distortion in the radial direction of the imaging lens, and p ₁ and p ₂ indicate lens distortion in the circumferential direction of the imaging lens. It is a coefficient. R is the radius of the imaging lens.

The spread in the space of the slit light SL emitted from the slit light source unit 11 can be regarded as a plane, and in that case, it can be represented by a plane expression such as Expression (5). In Expression (5), A _L , B _L , and C _L are parameters for defining a plane.

Further, since each of the stepped planes 101 to 104 of the calibration block 100 is a plane, it can be represented by a plane expression such as Expression (6). In Equation (6), A _C , B _C , and C _C are parameters for defining the planes of the stepped planes 101 to 104 of the calibration block 100.

  Here, in order to simplify the following description, q (bold body) = s · M (bold body) · W (bold body) · Q (bold body) in Expression (3) is expressed as Expression (7). Replace with

That is, in Expression (7), a matrix of 3 rows and 4 columns composed of elements of p _ij (1 ≦ i ≦ 3, 1 ≦ j ≦ 4) is expressed as s · M (bold body) · W (bold body). -Corresponds to Q (bold).

  The calibration for the three-dimensional coordinate conversion unit 25 to calculate Z (height) will be described. Expression (8) is an expression obtained by partially expanding Expression (7).

  Expression (9) is obtained from Expression (5) and Expression (8).

Equation (9) indicates that the reflection position at the same height Z is a straight line on the image plane, and the difference in y-intercept is proportional to the height Z. Therefore, the three-dimensional coordinate conversion unit 25 uses the calibration block 100 to obtain the relationship between the height Z and the position on the image plane (y-intercept) as calibration data. As a result, the three-dimensional coordinate conversion unit 25 can obtain an arbitrary height Z. In other words, the height Z _O of the measurement object is expressed by the following equation (10) when the y coordinate value is y _O , the height of the calibration block 100 is Z _C , and the y coordinate value is y _C. Can be represented.

  Next, calibration for the three-dimensional coordinate conversion unit 25 to calculate X and Y will be described. Expression (11) is obtained from Expression (8).

Next, calibration for correcting the movement of the measurement target object by the three-dimensional coordinate conversion unit 25 will be described. Although the world coordinates are defined at the place where the calibration block 100 is first installed, the movement direction of the measurement target object coincides with the X, Y, and Z axis directions of the world coordinates due to the movement of the mounting table 40. Not exclusively. Therefore, it is necessary to correct the movement amount and the movement direction.
Assuming that the movement pitch is M and the unit vector indicating the movement direction is (u _X , u _Y , u _Z ), the plane of the calibration block 100 shown in Expression (6) is translated, so Expression (6) The plane equation is expressed as equation (12).

Since the mounting table 40 only needs to move parallel to the bottom 110 (XY plane) of the calibration block 100, u _Z = 0, and the three-dimensional coordinate conversion unit 25 can obtain u _X and u _Y. it can. Further, the plane expression of Expression (12) is Expression (13).

  The relational expression of Expression (14) is derived from Expression (8) and Expression (13).

  When Expression (14) is converted into an expression for obtaining the three-dimensional coordinate values (X, Y, Z) of the world coordinates and simplified, it can be expressed as Expression (15).

In Equation (15), a matrix k (bold body) that is a matrix of 3 rows and 3 columns composed of elements of k _ij (1 ≦ i ≦ 3, 1 ≦ j ≦ 3) and elements s1, s2, and s3 A combination with a 3 × 1 matrix s (bold) is a transformation matrix set. Since x, y, M, X, Y, and Z are observables, it is desirable that the transformation matrix calculation unit 24 obtains the transformation matrix set statistically accurately with a plurality of samplings. For example, the transformation matrix calculation unit 24 obtains a plurality of transformation matrix sets by a plurality of samplings, and obtains an optimum value of the transformation matrix set by calculating an average value of each element. Then, the transformation matrix calculation unit 24 stores the obtained transformation matrix set in the storage unit 26.

The three-dimensional coordinate conversion unit 25 reads the transformation matrix set stored in the storage unit 26, and calculates the three-dimensional coordinate values (XYZ) of the world coordinate system of the calibration block 100 from arbitrary observation amounts x, y, M. ) To generate a plurality of combinations of two-dimensional coordinate values and three-dimensional coordinate values.
Next, the three-dimensional coordinate conversion unit 25 applies these generated combinations to Equation (9), Equation (11), and Equation (13) to perform calibration for performing calibration between the image plane and the real space. Data is generated.

  Next, the operation in the normal measurement mode in which the three-dimensional shape measurement apparatus 1 performs actual three-dimensional shape measurement using the measurement target object will be described. FIG. 7 is a flowchart showing a processing procedure of three-dimensional shape measurement executed by the three-dimensional shape measuring apparatus 1. After the measurement target object is placed on the placement table 40, when the three-dimensional shape measurement apparatus 1 gives an operation start instruction for the normal measurement mode to the measurement control unit 21, the process of the flowchart shown in FIG. .

  Since the processing from step S701 to step S705 is the same as the processing from step S501 to step S505 described above, description thereof is omitted.

  In step S <b> 706, the reflection position calculation unit 23 detects a feature point from the light section line detected by the light section line detection unit 22. Next, in step S <b> 707, the three-dimensional coordinate conversion unit 25 calculates Z of the measurement target object by executing the calculation of Expression (10) described above. Next, in step S708, the three-dimensional coordinate conversion unit 25 calculates X and Y of the measurement target object by executing the calculation of the above-described equation (11). Next, in step S709, the three-dimensional coordinate conversion unit 25 reads the calibration data stored in the storage unit 26, and corrects the three-dimensional coordinate value (XYZ) according to the movement pitch M of the measurement target object. I do.

  Next, in step S710, the measurement control unit 21 determines whether or not the measurement target object has been scanned with the bright line LL. If it is determined that the scanning has been completed (S710: YES), the process of this flowchart is terminated. If it is determined that scanning has not ended (S710: NO), the process returns to step S704.

  As described above, the three-dimensional shape measurement apparatus 1 according to an embodiment of the present invention uses the world coordinate system based on the feature point coordinate values of the feature points obtained by imaging the calibration block and the movement pitch of the moving unit. Since the transformation matrix to convert to the 3D coordinate value of the feature point is generated and the calibration data is generated, there is no need to measure the entire calibration block reliably as in the past, and the calibration in the height direction is efficient. And with high accuracy.

In the present embodiment, the example in which the mounting table 40 is moved with respect to the fixed optical measurement unit 10 has been described. However, in addition to this, the optical measurement unit 10 is moved in the transport direction A with respect to the fixed mounting table 40. You may make it move to a reverse direction. For example, when measuring a measurement target object having a large mass, the drive unit can be reduced in size and cost by moving the slit light source unit 11 rather than moving the mounting table 40.
Further, both the mounting table 40 and the optical measurement unit 10 may be moved.

  Further, the slit light SL emitted from the slit light source unit 11 can be, for example, halogen light in addition to the laser light. The slit light source unit 11 that emits the slit light by the halogen light can emit the slit light having a large light amount.

In addition, some or all of the functions of the three-dimensional shape measuring apparatus 1 may be incorporated into a robot capable of performing operations such as expansion, contraction, extension, and turning of the arm unit and the hand unit. This robot is, for example, an industrial robot having a degree of freedom of three or more axes and having a movable arm portion and hand portion. A more specific example of this robot will be described.
FIG. 8 is a perspective view of the robot, the calibration block, and the mounting table showing how the robot incorporating a part of the functions of the three-dimensional shape measuring apparatus 1 performs calibration. As shown in the figure, the robot 8 includes a support base 81 fixed to the ground, an arm portion 82 capable of turning and bending, and a hand portion 83 capable of rotating and swinging. The A frame 84 on which the slit light source unit 11 and the imaging unit 12 are fixedly supported is attached to the hand unit 83. The calibration block 100 is mounted on a mounting table 40 fixed to the ground.
The robot 8 operates the arm unit 82 and the hand unit 83 in a complex manner under the control of a robot controller (not shown), and moves the slit light source unit 11 and the imaging unit 12 in the movement direction B. The support base 81 may be installed at a place fixed to the ground such as a wall or ceiling in addition to the ground.

  Further, the slit light source unit 11 and the imaging unit 12 are fixedly installed as in the present embodiment, and the mounting table 40 is attached to the hand unit 83 of the robot 8 so that the mounting surface is horizontal with respect to the ground. Then, it may be moved in the direction opposite to the moving direction B.

  Further, the imaging unit 12 may be attached to the hand unit 83 of the robot 8 so as to freely move in the space. By making the imaging unit 12 freely movable in this manner, it is possible to perform three-dimensional shape measurement without the blind spot of the measurement target object.

  Moreover, you may make it implement | achieve a part of the three-dimensional shape measuring apparatus which is this embodiment, for example, the function of a control part with a computer. In this case, a program for realizing the function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read into a computer system and executed. Here, the “computer system” includes an OS (Operating System) and peripheral hardware. The “computer-readable recording medium” refers to a portable recording medium such as a flexible disk, a magneto-optical disk, an optical disk, and a memory card, and a storage device such as a hard disk built in the computer system. Furthermore, a “computer-readable recording medium” dynamically holds a program for a short time, like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. It is also possible to include one that holds a program for a certain period of time, such as a volatile memory inside a computer system that becomes a server or client in that case. In addition, the above program may be for realizing a part of the above-described functions, and further, may be realized by combining the above-described functions with a program already recorded in the computer system. .

  As mentioned above, although embodiment of this invention was explained in full detail with reference to drawings, the concrete structure is not restricted to this embodiment, The design etc. of the range which does not deviate from the summary of this invention are included.

DESCRIPTION OF SYMBOLS 1 3D shape measuring apparatus 8 Robot 10 Optical measurement part 11 Slit light source part 12 Imaging part 20 Control part 21 Measurement control part 22 Optical cutting line detection part 23 Reflection position calculation part (feature point information generation part)
24 conversion matrix calculation unit 25 three-dimensional coordinate conversion unit 26 storage unit 30 mounting table driving unit 40 mounting table 81 support table 82 arm unit 83 hand unit 84 frame 100 calibration block A transport direction B moving direction SL slit light LL bright line RL reflected light

Claims (6)

A light source unit that emits slit light;
A mounting table on which a calibration block is mounted;
An imaging unit that captures an optical cutting line of the slit light applied to the calibration block;
A moving unit that moves at least one of the mounting table and the combination of the light source unit and the imaging unit;
An optical cutting line detection unit for detecting the optical cutting line from a captured image captured by the imaging unit;
Feature point information is calculated by detecting a feature point from the light section line, calculating a feature point coordinate value, and associating the feature point coordinate value, the movement pitch of the moving unit, and predetermined attribute data of the calibration block. A feature point information generation unit to generate,
A transformation matrix calculator for calculating a transformation matrix for transforming the feature point coordinate value and the moving pitch into a three-dimensional coordinate value of the feature point in the world coordinate system;
Based on the feature point information and the transformation matrix, calibration data is obtained by obtaining a combination of a two-dimensional coordinate value of the image plane of the captured image and a three-dimensional coordinate value of the world coordinate system corresponding to the two-dimensional coordinate value. A three-dimensional coordinate conversion unit to be generated;
A three-dimensional shape measuring apparatus comprising:   The three-dimensional shape measurement apparatus according to claim 1, wherein the light section line detection unit detects the light section line by obtaining a barycentric position of luminance from the captured image. The calibration block has a plurality of plane portions having different height dimensions from the bottom portion placed on the mounting table, and a light cutting line generated by irradiation of the slit light on each of the plurality of plane portions. 3. The three-dimensional shape measuring apparatus according to claim 1, wherein the three-dimensional shape measuring apparatus has a non-parallel straight outline, and a pattern having different light reflectivity is formed at the outline. The feature point information generation unit calculates a vertex coordinate value of the pattern based on a plurality of the feature point coordinate values, the vertex coordinate value, the movement pitch of the moving unit, and predetermined attribute data of the calibration block The three-dimensional shape measuring apparatus according to claim 3, wherein vertex information that associates with each other is generated and added to the feature point information. In the calibration method of the three-dimensional shape measuring apparatus,
Move at least one of a combination of a mounting table on which a calibration block is mounted, a light source unit that emits slit light, and an imaging unit that captures an optical cutting line of the slit light that is irradiated onto the calibration block. Steps,
Detecting the optical cutting line from a captured image captured by the imaging unit;
Feature point information is calculated by detecting a feature point from the light section line, calculating a feature point coordinate value, and associating the feature point coordinate value, the movement pitch of the moving unit, and predetermined attribute data of the calibration block. Generating step;
Calculating a transformation matrix for converting the feature point coordinate value and the movement pitch into a three-dimensional coordinate value of the feature point in the world coordinate system;
Based on the feature point information and the transformation matrix, calibration data is obtained by obtaining a combination of a two-dimensional coordinate value of the image plane of the captured image and a three-dimensional coordinate value of the world coordinate system corresponding to the two-dimensional coordinate value. Generating step;
A method for calibrating a three-dimensional shape measuring apparatus, comprising: A light source unit that emits slit light;
A mounting table on which a calibration block is mounted;
An imaging unit that captures an optical cutting line of the slit light applied to the calibration block;
A hand unit to which any one of the mounting table and the combination of the light source unit and the imaging unit is attached;
An arm part to which the hand part is movably attached;
An optical cutting line detection unit for detecting the optical cutting line from a captured image captured by the imaging unit;
A feature point is calculated by detecting a feature point from the light section line, and calculating a feature point coordinate value, a movement pitch in one direction of the hand unit, and predetermined attribute data of the calibration block A feature point information generation unit for generating point information;
A transformation matrix calculator for calculating a transformation matrix for transforming the feature point coordinate value and the moving pitch into a three-dimensional coordinate value of the feature point in the world coordinate system;
Based on the feature point information and the transformation matrix, calibration data is obtained by obtaining a combination of a two-dimensional coordinate value of the image plane of the captured image and a three-dimensional coordinate value of the world coordinate system corresponding to the two-dimensional coordinate value. A three-dimensional coordinate conversion unit to be generated;
A robot characterized by comprising:

Images