JP2016011930A - Three-dimensional data connection method, measurement method, measurement apparatus, structure manufacturing method, structure manufacturing system, and shape measurement program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To efficiently perform a connection of three-dimensional data.SOLUTION: A three-dimensional data connection method of connecting a plurality of three-dimensional data to be acquired for a measurement object includes: displaying a live view image of the measurement object; displaying one part of an image of the measurement object to be photographed upon acquiring first three-dimensional data so as to overlap with the live view image as an overlay image; changing a display state of the overlay image; and connecting second three-dimensional data on the measurement object having a part overlapping with the first three-dimensional data and corresponding to the live view image to the first three-dimensional data.

Description

  The present invention relates to a method for connecting three-dimensional data, a measuring method, a measuring apparatus, a structure manufacturing method, a structure manufacturing system, and a shape measuring program.

  As a technique for measuring the three-dimensional shape of the measurement object, for example, a phase shift method is known. A shape measuring apparatus using the phase shift method includes a projection unit, an imaging unit, and a control unit. This projection unit projects a striped pattern light having a sinusoidal light intensity distribution (hereinafter referred to as structured light) onto a measurement object. At this time, when the four types of structured light having different phases are projected onto the measurement object, the imaging unit images each measurement object and acquires four phase images. The control unit applies data relating to the signal intensity of each pixel in the four images captured by the imaging unit to a predetermined arithmetic expression, and obtains the phase value of the fringes at each pixel according to the surface shape of the measurement target. Then, the calculation unit calculates three-dimensional data (for example, point cloud data) of the measurement object from the phase value of the stripes in each pixel using the principle of triangulation. An apparatus using this phase shift method is disclosed in Patent Document 1, for example.

  In the above case, for example, when the measurement object does not fit in the imaging field of the imaging unit, the three-dimensional data of the entire measurement object is connected by connecting the three-dimensional data calculated by measuring different positions of the measurement object. The shape can be measured. The joining of the three-dimensional data is performed, for example, by overlapping a part of the three-dimensional data. Moreover, when superimposing a part of three-dimensional data, the three-dimensional data in the same area | region of a measuring object are superimposed.

  In the process of superimposing parts of the three-dimensional data, first, an image of the first part of the measurement object is captured, and then an image of the second part is captured so that the image of the first part partially overlaps, Three-dimensional data is calculated for each. The range of the overlapping portion between the first portion image and the second portion image is set in advance. For example, the range of the overlapping part is set so that the first part image and the second part image include three or more common feature areas. This feature area is an area in each image of the first part image and the second part image, for example, and can be identified by the change in luminance (signal intensity) with respect to the other areas. is there. In this case, the change in luminance (signal intensity) is based on a change in the shape of the measurement object, the light reflectance of the surface, and the like. Then, in the three-dimensional data of the first part and the second part, the three-dimensional data of the first part and the three-dimensional data of the second part are connected so that the parts captured in an overlapping manner are overlapped. At this time, the pixel of the above three feature regions that are common coordinate data in the three-dimensional data of the first part and the three-dimensional data of the second part is searched, and an overlapping part of the first part and the second part is found. to decide. When three-dimensional data is connected a plurality of times, new three-dimensional data is sequentially overlapped.

US Pat. No. 5,450,204

  However, in the past, it was difficult to capture images while confirming how much the images of the first part and the second part overlap. It was difficult to perform the connection efficiently.

  In view of the circumstances as described above, an object of the present invention is to efficiently link three-dimensional data.

  According to the first aspect of the present invention, there is provided a three-dimensional data connection method for connecting a plurality of three-dimensional data acquired for a measurement object, wherein a live view image of the measurement object is displayed; Displaying a part of the image of the measurement object imaged when acquiring the three-dimensional data on the live view image as an overlay image, changing the display state of the overlay image, and the first tertiary There is provided a method for connecting three-dimensional data, including connecting the second three-dimensional data of the measurement object corresponding to the live view image and the first three-dimensional data having a portion overlapping with the original data. The

  According to the second aspect of the present invention, there is provided a measurement method for measuring a three-dimensional shape of a measurement object, wherein a plurality of three-dimensional data about the measurement object is obtained, and a tertiary according to the first aspect of the present invention. There is provided a measuring method including connecting a plurality of three-dimensional data using a connecting method of original data, and calculating a three-dimensional shape of a measurement object based on the connected result.

  According to the third aspect of the present invention, there is provided a measuring device for measuring a three-dimensional shape of a measurement object, an imaging unit that images the measurement object, a display unit that displays an image captured by the imaging unit, An operation for causing the imaging unit to capture a live view image of the measurement object and displaying the live view image on the display unit, and a part of the image of the measurement object when acquiring the first three-dimensional data of the measurement object An operation of causing the imaging unit to capture a part of the image of the measurement object as an overlay image and displaying the image on the display unit on the live view image; an operation of changing a display state of the overlay image; and the first three-dimensional data Based on the result of the connection, the control unit for performing the operation of connecting the second three-dimensional data of the measurement object corresponding to the live view image and the first three-dimensional data having an overlapping portion. Measurement object Measuring device and a calculation unit for calculating a three-dimensional shape is provided.

  According to the fourth aspect of the present invention, the design information relating to the shape of the structure is produced, the structure is produced based on the design information, and the shape of the produced structure is measured. A shape manufacturing method according to the second aspect, and a structure manufacturing method including comparing shape information related to the shape of the structure obtained by the shape measuring method with design information are provided.

  According to the fifth aspect of the present invention, a design apparatus for producing design information related to the shape of the structure, a molding apparatus for producing the structure based on the design information, and the present invention for measuring the shape of the produced structure. There is provided a structure manufacturing system including a measuring apparatus according to the third aspect of the present invention, and an inspection apparatus that compares design information with shape information related to the shape of the structure obtained by the measuring apparatus.

  According to the sixth aspect of the present invention, processing for displaying a live view image of a measurement object on a computer included in a measurement apparatus that measures the three-dimensional shape of the measurement object, and first three-dimensional data are acquired. A process for displaying a part of the image of the measurement object imaged on the live view image as an overlay image, a process for changing the display state of the overlay image, and a part overlapping the first three-dimensional data. There is provided a shape measurement program for executing a process of connecting the second three-dimensional data of the measurement object corresponding to the live view image and the first three-dimensional data.

  According to the aspect of the present invention, it is possible to efficiently link three-dimensional data.

It is a figure which shows an example of the shape measuring apparatus which concerns on 1st Embodiment. It is a block diagram which shows an example of a detailed structure of the shape measuring apparatus shown in FIG. It is a figure which shows intensity distribution of structured light in a projection area. It is a figure which shows the state by which the fringe pattern of each phase was projected on the plane without a measuring object. It is a flowchart explaining an example of a measuring method, explaining operation | movement of a shape measuring apparatus. It is a figure which shows an example of the positional relationship of a measuring object and a shape measuring apparatus. It is a figure which shows the image displayed on a display apparatus in the measuring method which concerns on this embodiment. It is a figure which shows the relationship between a reference image and an overlay image. It is a figure which shows the image displayed on a display apparatus in the measuring method which concerns on this embodiment. It is a figure which shows the image displayed on a display apparatus in the measuring method which concerns on this embodiment. It is a figure which shows the image displayed on a display apparatus in the measuring method which concerns on 2nd Embodiment. It is a figure which shows the image displayed on a display apparatus in the measuring method which concerns on 2nd Embodiment. It is a figure which shows the image displayed on a display apparatus in the measuring method which concerns on 3rd Embodiment. It is a figure which shows the other example of the image displayed on a display apparatus. It is a figure which shows an example of the shape measuring apparatus which concerns on a modification. It is a figure which shows the image displayed on the display apparatus of the shape measuring apparatus which concerns on a modification. It is a block diagram which shows an example of embodiment of a structure manufacturing system. It is a flowchart which shows an example of embodiment of a structure manufacturing method.

<First Embodiment>
FIG. 1 is a diagram illustrating an example of a shape measuring apparatus according to the first embodiment. In FIG. 1, the right direction of the drawing is the X axis, a certain direction orthogonal to the X axis is the Y axis, and a direction orthogonal to the X axis and the Y axis is the Z axis. The shape measuring device 1 is a device that measures the three-dimensional shape of the measuring object 2 using the phase shift method. As shown in FIG. 1, the shape measuring apparatus 1 includes a projection unit 10, an imaging unit 50, an arithmetic processing unit 60, a display device 70, and a housing 90. The shape measuring apparatus 1 has a configuration in which the projection unit 10, the imaging unit 50, the arithmetic processing unit 60, and the display device 70 are accommodated in a portable case 90.

  The projection unit 10 generates a line-shaped projection light 100 having a light intensity distribution along the first direction D1 (X-axis direction in FIG. 1). Then, the projection unit 10 scans the generated projection light 100 along a second direction D2 (the Y-axis direction in FIG. 1) different from the first direction, thereby causing the structured light 101 with respect to the projection region 200. Project. The structured light 101 of the first embodiment is structured light used in the phase shift method. Details of the structured light 101, the projection region 200, and the feature region will be described later (see FIGS. 3 and 4).

  As shown in FIG. 1, the projection unit 10 includes a light generation unit 20, a projection optical system 30, and a scanning unit 40. The light generation unit 20 generates the projection light 100. The projection optical system 30 projects the projection light 100 generated by the light generation unit 20. The projection light 100 emitted from the projection optical system 30 is projected toward the measurement object 2 or the vicinity of the measurement object 2 via the scanning unit 40. The scanning unit 40 scans the projection light 100 in the second direction D2 (Y-axis direction in FIG. 1).

  The imaging unit 50 images the measurement object 2. The imaging unit 50 is disposed at a position different from the position of the projection unit 10. The imaging unit 50 images the measurement object 2 onto which the projection light 100 is projected from a direction different from the direction in which the projection unit 10 projects. The imaging unit 50 captures an image of the measurement object 2 onto which the structured light 101 is projected (hereinafter referred to as “measurement image”), for example. In addition, the imaging unit 50 captures an image of the measurement object 2 using natural light (hereinafter referred to as “reference image”), for example. The reference image includes a still image and a live view image.

  The imaging unit 50 includes a light receiving optical system 51 and an imaging device 52. As the imaging unit 50, a monochrome camera is used. The light receiving optical system 51 is an optical system that causes the imaging device 52 to form an image of a region including a portion on which the projection light 100 is projected on the surface of the measurement object 2. For the light receiving optical system 51, for example, a plurality of lenses are used. The imaging device 52 generates image data of the measurement object 2 based on the image formed by the light receiving optical system 51 and stores the generated image data.

  The arithmetic processing unit 60 controls the generation of the projection light 100 by the light generation unit 20. In addition, the arithmetic processing unit 60 controls the scanning unit 40 and the imaging unit 50 so that the scanning of the projection light 100 by the scanning unit 40 and the imaging of the measurement object 2 by the imaging unit 50 are synchronized. In addition, the arithmetic processing unit 60 controls the imaging unit 50 so as to capture an image of the measurement target 2 by natural light. Further, the arithmetic processing unit 60 calculates the three-dimensional shape of the measurement object 2 based on the luminance data (signal intensity) of each pixel in the image data captured by the imaging unit 50.

  Next, detailed configurations of the projection unit 10, the imaging unit 50, and the arithmetic processing unit 60 included in the shape measuring apparatus 1 will be described with reference to FIG. FIG. 2 is a block diagram showing an example of a detailed configuration of the shape measuring apparatus 1 shown in FIG. When a three-axis coordinate system is set as shown in FIG. 1, in FIG. 2, the right direction of the paper is the X axis, the upward direction of the paper is the Z axis, and the direction from the back of the paper to the front is the Y axis. Become. As shown in FIG. 2, the projection unit 10 includes a laser controller 21, a laser diode (light source) 22, a projection optical system 30, and a scanning unit 40. The light generation unit 20 illustrated in FIG. 1 includes a laser controller 21 and a laser diode 22.

  The laser controller 21 controls irradiation of the laser light by the laser diode 22 based on a command signal from the control unit 62. For example, the laser controller 21 can change the light intensity of the laser light so as to be synchronized with the scanning of the projection light 100 by the scanning unit 40 based on a command signal from the control unit 62, as will be described later. ing. The control unit 62 will be described later. The laser diode 22 is a light source that emits laser light based on a control signal from the laser controller 21. The laser diode 22 includes, for example, a red laser diode that emits red light, a green laser diode that emits green light, and a blue laser diode that emits blue light.

  The projection optical system 30 projects the projection light 100 as described above. The projection optical system 30 includes one or a plurality of transmission optical elements or reflection optical elements.

  The scanning unit 40 reflects the projection light 100 emitted from the projection optical system 30 by using, for example, a reflection optical element such as a mirror, and changes the reflection angle thereof to change the projection light 100 in the second direction D2 ( Scan in the Y-axis direction in FIG. As an example of the reflective optical element constituting the scanning unit 40, a MEMS (Micro Electro Mechanical Systems) mirror that resonates the mirror with static electricity and changes the reflection angle of the projection light 100 is used. The second direction D2 is a direction on the measurement object 2 different from the first direction D1 (X-axis direction in FIG. 2). For example, the first direction D1 and the second direction D2 are orthogonal to each other.

  As shown in FIG. 1, the MEMS mirror vibrates in the direction S (see FIG. 1) with the vibration center AX in the paper as an axis, and changes the reflection angle while reflecting the projection light 100 at a predetermined reflection angle. The scanning width in the second direction D2 by the MEMS mirror (that is, the length in the second direction D2 in the projection region 200) is determined by the amplitude in the vibration direction S of the MEMS mirror. Further, the speed at which the projection light 100 is scanned in the second direction D2 by the MEMS mirror is determined by the angular speed (that is, the resonance frequency) of the MEMS mirror. Further, by vibrating the MEMS mirror, the projection light 100 can be scanned back and forth. The start position of scanning with the projection light 100 is arbitrary. For example, in addition to starting the scanning of the projection light 100 from the end of the projection area 200, the scanning may be started from approximately the center of the projection area 200.

  FIG. 3 is a diagram illustrating the intensity distribution of the structured light 101 in the projection region 200. When a three-axis coordinate system as shown in FIG. 1 is set, in FIG. 3, the right direction of the paper surface is the Y axis, the upward direction of the paper surface is the X axis, and the direction from the back of the paper surface to the front is the Z axis. Become.

  As shown in FIG. 3, the projection light 100 is slit-shaped light having a predetermined length in the first direction D1. The projection light 100 is scanned over a predetermined distance in the second direction D2, thereby forming a rectangular projection region 200. The projection area 200 is an area onto which the structured light 101 is projected, and is an area defined by the first direction D1 and the second direction D2. The projection area 200 includes part or all of the measurement object 2.

  The structured light 101 shown in FIG. 3 is pattern light having a periodic light intensity distribution along the second direction D2. In the first embodiment, a stripe pattern P having a sinusoidal periodic light intensity distribution along the second direction D2 is used as an example of the structured light 101. The fringe pattern P is formed, for example, by setting the wavelength of the projection light 100 to a predetermined wavelength (eg, about 680 nm) and scanning in the second direction D2 while periodically changing the light intensity of the projection light 100. The stripe pattern P has a light-dark pattern in which a bright part (white part in FIG. 3) and a dark part (black part in FIG. 3) change along the second direction D2. The fringe pattern P is also expressed as a shading pattern in which a dark portion (black portion in FIG. 3) and a thin portion (white portion in FIG. 3) gradually change. Further, since the stripe pattern P is a lattice pattern, it is also expressed as a lattice pattern. Further, the second direction D2 is also referred to as a light / dark direction, a light / dark direction, or a lattice direction.

  Subsequently, as shown in FIG. 2, the imaging unit 50 includes a light receiving optical system 51, a CCD camera 52a, and an image memory 52b. The imaging device 52 includes a CCD camera 52a and an image memory 52b. As described above, the light receiving optical system 51 forms an image of a region including the portion on which the projection light 100 is projected or an image of natural light on the surface of the measurement object 2 on the light receiving surface of the CCD camera 52a. The CCD camera 52a is a camera using a charge coupled device.

  Image data generated by the CCD camera 52a is composed of signal intensity data for each pixel. For example, the image data is composed of signal intensity data of 512 × 512 = 262144 pixels. The image memory 52b stores image data generated by the CCD camera 52a.

Subsequently, as shown in FIG. 2, the calculation processing unit 60 includes an operation unit 61, a control unit 62, a setting information storage unit 63, a capture memory 64, a calculation unit 65, an image storage unit 66, and a display control unit 67. Have.
The operation unit 61 outputs an operation signal corresponding to a user operation to the control unit 62. The operation unit 61 is, for example, a button, switch, touch panel, or the like operated by the user.

  The control unit 62 controls the light generation unit 20, the scanning unit 40, and the imaging unit 50. The control unit 62 executes the following control according to the program stored in the setting information storage unit 63.

  The control unit 62 outputs a command signal to the scanning unit 40 and the CCD camera 52a, and controls the imaging of the measurement object 2 by the CCD camera 52a to be synchronized with the scanning of the fringe pattern P by the scanning unit 40. Further, the control unit 62 performs control so as to synchronize imaging of one frame by the CCD camera 52a and a plurality of times of scanning of the stripe pattern P. The control unit 62 can control the CCD camera 52a independently. In this case, under the control of the control unit 62, the CCD camera 52a captures an image of the measuring object 2 with natural light at a predetermined frame rate.

  The control unit 62 can emit desired laser light combining red light, blue light, and green light from the laser diode 22 by outputting a command signal to the laser controller 21. The control unit 62 can adjust the light intensity of the laser light emitted from the laser diode 22 by outputting a command signal to the laser controller 21. When projecting the fringe pattern P onto the measurement object 2, the control unit 62 periodically changes the light intensity of the projection light 100 having a predetermined wavelength by synchronously controlling the laser controller 21 and the scanning unit 40, for example. The projection light 100 is scanned in the second direction D2.

  The frequency of the MEMS mirror that constitutes the scanning unit 40 is set to, for example, 500 Hz (the oscillation cycle of the MEMS mirror is 2 ms for reciprocation). The shutter speed of the CCD camera 52a (exposure time of the CCD camera 52a) is set to 40 ms, for example. Accordingly, while the CCD camera 52a captures one image, the scanning unit 40 scans the projection light 100 in the projection region 200 40 times (20 reciprocating scans). The control unit 62 performs control so that the projection light 100 is reciprocated 20 times, for example, by the scanning unit 40 during imaging of one frame by the CCD camera 52a. However, it is possible to arbitrarily set how many reciprocating scans of the projection light 100 are performed when the CCD camera 52a captures one frame. For example, by adjusting the shutter speed of the CCD camera 52a and the frequency of the MEMS mirror, the number of scans of the projection light 100 captured by one frame imaging is adjusted.

  The setting information storage unit 63 stores a program for causing the control unit 62 to execute control. The setting information storage unit 63 stores a program for causing the display control unit 67 to execute display control. As such a program, for example, a program for displaying a live view image on the display device 70 or a part of a reference image of the measurement object 2 is displayed on the display device 70 as an overlay image so as to be superimposed on the live view image. Programs. The live view image is an image in which the output from the CCD camera 52a is displayed on the display device 70 in real time. The live view image is used for the user to confirm an image of the measurement object 2 or the like captured by the imaging unit 50, for example. The overlay image is an image displayed on the live view image.

  In addition, the setting information storage unit 63 stores a program for causing the calculation unit 65 to execute calculation processing of a three-dimensional shape. The setting information storage unit 63 also stores calibration information used when calculating the actual coordinate value of the measurement object 2 from the fringe phase of the fringe pattern P in the calculation process of the calculation unit 65.

  The capture memory 64 captures and stores the image data stored in the image memory 52b. The capture memory 64 stores a measurement image of the measurement object 2 imaged by projecting the fringe pattern P, a reference image of the measurement object 2 by natural light, and the like. The capture memory 64 is provided with a plurality of storage areas. The image data of the measurement image and the image data of the reference image are stored in different storage areas, for example.

  The calculation unit 65 performs a predetermined calculation according to the program and calibration information stored in the setting information storage unit 63. The computing unit 65 detects the feature region of the measurement object 2 by image processing based on the reference image imaged by the imaging unit 50, for example. The feature region is a region that is included in the reference image of the measurement object 2, for example, and can be identified by the change in luminance with respect to other regions. In this case, the change in luminance is based on a change in the shape of the measurement object 2, the light reflectance of the surface, and the like. In addition, when such a marker (not shown) is arranged on or around the measurement object 2 as such a feature region, the marker may be included as a feature region.

  The image storage unit 66 stores the three-dimensional shape data of the measurement object 2 calculated by the calculation process by the calculation unit 65. The image storage unit 66 stores the image data of the measurement object 2 calculated by the calculation unit 65, the image data used in the calculation process, and the like. Such image data includes still images, moving images, live view images, and the like.

  The display control unit 67 performs display control of the live view image according to the program stored in the setting information storage unit 63. In this case, the display control unit 67 reads the live view image of the measurement object 2 stored in the image storage unit 66 and displays it on the display screen of the display device 70.

  In addition, the display control unit 67 performs display control of the overlay image according to a program stored in the setting information storage unit 63. In this case, the display control unit 67 reads the reference image data of the measurement object 2 stored in the image storage unit 66. Then, the display control unit 67 displays a part of the read reference image on the display screen of the display device 70 as an overlay image superimposed on the live view image. The range in which the overlay image is displayed on the display screen may be set by the user in advance or may be set automatically.

  In addition, the display control unit 67 performs control to change the display state of the overlay image displayed on the display screen. The change of the display state of the overlay image includes, for example, changing the transparency of the overlay image. About the transparency of an overlay image, the display control part 67 may set automatically, and the display control part 67 may set according to the value set by the user. When the user sets the transparency of the overlay image, the display control unit 67 may perform control to display information indicating the transparency on the display screen of the display device 70.

  Further, the display control unit 67 executes display control of a three-dimensional image according to a program stored in the setting information storage unit 63. That is, the display control unit 67 reads the three-dimensional shape data stored in the image storage unit 66 in accordance with the operation of the operation unit 61 by the user or automatically. And the display control part 67 performs control which displays the image of the three-dimensional shape of the measuring object 2 on the display screen of the display apparatus 70 based on the read-out three-dimensional shape data.

  The display device 70 is a device that displays a three-dimensional image or a reference image (a still image and a live view image) of the measurement target 2 under the control of the display control unit 67. For example, a touch panel is used as the display device 70. The display device 70 is for displaying an image or the like, but can also be operated by a touch panel, and is also used as the operation unit 61.

  The control unit 62, the calculation unit 65, and the display control unit 67 are configured by a calculation processing device such as a CPU (Central Processing Unit). That is, the arithmetic processing unit performs processing executed by the control unit 62 in accordance with a program stored in the setting information storage unit 63. In addition, the arithmetic processing unit performs processing executed by the arithmetic unit 65 in accordance with a program stored in the setting information storage unit 63. Further, the arithmetic processing unit performs processing executed by the display control unit 67 in accordance with a program stored in the setting information storage unit 63. This program includes a measurement program.

  This measurement program acquires processing for displaying a live view image of the measurement object 2 and three-dimensional data (first three-dimensional data) of the measurement object 2 on the arithmetic processing unit (control unit 62). A part of the image of the measurement object 2 that is picked up at the time is displayed as an overlay image superimposed on the live view image, a process of changing the display state of the overlay image, and a part overlapping the first three-dimensional data And the second three-dimensional data of the measurement object 2 corresponding to the live view image and the process of linking the first three-dimensional data are performed.

Next, the principle of the phase shift method will be described.
The phase shift method is based on the principle of triangulation, and a fringe image (the fringe pattern P is projected by shifting the fringe phase of the fringe pattern P having a sinusoidal light intensity distribution projected onto the measurement object 2. This is a method of measuring the shape three-dimensionally by analyzing the measured image of the measured object 2). In the present embodiment, the fringe pattern P is four types of fringe patterns P obtained by shifting the fringe phase by π / 2 along the second direction D2. Here, the phase of the fringe pattern P can be rephrased as a phase of a sine wave that is a light intensity distribution of the fringe pattern P. That is, four types of fringe patterns P are generated by shifting a sine wave, which is a light intensity distribution, by π / 2 along the second direction D2.

  Hereinafter, for example, the reference stripe pattern P is a first stripe pattern (first phase light) P1, and the phase of the first stripe pattern P1 is zero. The stripe pattern P obtained by shifting the phase of the first stripe pattern P1 by π / 2 is defined as the second stripe pattern (second phase light) P2, and the stripe pattern obtained by shifting the phase of the first stripe pattern P1 by π. Let P be the third stripe pattern (third phase light) P3, and let the stripe pattern P obtained by shifting the phase of the first stripe pattern P1 by 3π / 2 be the fourth stripe pattern (fourth phase light) P4.

  5A to 5D are views showing a state in which the first stripe pattern P1 to the fourth stripe pattern P4 are projected on a plane without the measurement object 2, and the imaging region 210 in the projection region 200 is shown. It is an image. Note that the imaging region 210 is a region of the measurement object 2 that is imaged by the imaging unit 50. 5A shows the first stripe pattern P1, FIG. 5B shows the second stripe pattern P2, FIG. 5C shows the third stripe pattern P3, and FIG. 5D shows the fourth stripe pattern P4.

  In the phase shift method, the first fringe pattern P1 to the fourth fringe pattern P4 as shown in FIGS. 5A to 5D are projected from the projection unit 10 onto the measurement object 2 and are different from the projection unit 10. The measurement object 2 is imaged by the imaging unit 50 arranged at an angle. At this time, the projection unit 10, the measurement object 2, and the imaging unit 50 are arranged so as to have a triangulation positional relationship.

  The imaging unit 50 captures the measurement object 2 and obtains four measurement images in a state where the first stripe pattern P1 to the fourth stripe pattern P4 are respectively projected onto the measurement object 2. Then, the arithmetic processing unit 60 applies the data on the signal strengths of the four measurement images captured by the imaging unit 50 to the following (Equation 1), and the fringes in each pixel according to the surface shape of the measurement object 2 are calculated. A phase value φ is obtained.

φ (u, v) = tan ⁻¹ {(I4 (u, v) −I2 (u, v)) / (I1 (u, v) −I3 (u, v))} (Expression 1)
However, (u, v) indicates the position coordinates of the pixel. I1 is the signal intensity of the measurement image captured when the first fringe pattern P1 is projected. Similarly, I2 is the second stripe pattern P2, I3 is the third stripe pattern P3, and I4 is the signal intensity of the measurement image when the fourth stripe pattern P4 is projected.

  In this way, the phase of the signal intensity that changes sinusoidally for each pixel of the image can be obtained. A line (equal phase line) obtained by connecting points having the same phase φ (u, v) represents the shape of a cross section obtained by cutting an object along a certain plane in the same manner as the cutting line in the optical cutting method. Therefore, a three-dimensional shape (height information at each point of the image) is obtained by the principle of triangulation based on this phase φ (u, v).

  As shown in FIGS. 5A to 5D, every time the phase of the fringe pattern P shifts to 0, π / 2, π, 3π / 2, the position of the fringe on the imaging region 210 (the fringe pattern). It is confirmed that the bright portions and the dark portions of the stripes are shifted by the phase difference. As described above, by shifting the phase of the fringe pattern P, the bright and dark portions of the fringe are projected in the second direction D2 on the imaging region 210 in accordance with the phase. The

  As shown in FIGS. 5A to 5D, for example, in the second stripe pattern P2, the second direction D2 is a distance corresponding to the phase π / 2 of the stripe position with respect to the first stripe pattern P1. It is shifted to. Further, in the third stripe pattern P3, the position of the stripe is shifted in the second direction D2 by a distance corresponding to the phase π with respect to the first stripe pattern P1. Similarly, in the fourth stripe pattern P4, the position of the stripe is shifted in the second direction D2 by a distance corresponding to the phase 3π / 2 with respect to the first stripe pattern P1. For this reason, on the imaging region 210, the position of the stripe is projected from the first stripe pattern P1 to the fourth stripe pattern P4 in the second direction D2 at equal intervals.

  5A to 5D show an image of the stripe pattern P projected on the plane, the shape of the image of the stripe pattern P is not changed. When the measuring object 2 is present, the fringe pattern P is projected on the surface of the measuring object 2, so that the image of the fringe pattern P in the second direction D2 (see FIG. 3 in the Y-axis direction).

  Next, an example of a measuring method using the shape measuring apparatus 1 configured as described above will be described. In the first embodiment, a case where the measurement object 2 is arranged so as to be larger than the imaging field of view of the imaging unit 50 will be described as an example. For example, as shown in FIG. 1, the measurement object 2 has a size larger than the projection region 200 in the Y direction. In this case, the first part 2M on the −Y side and the second part 2N on the + Y side (see FIG. 1) of the measurement object 2 are respectively measured, and the three-dimensional data are connected to each other to measure the entire measurement object 2. A procedure for measuring a three-dimensional shape is given as an example. In the process of joining parts of the three-dimensional data, the image of the second part 2N is captured so that the part of the image of the first part 2M overlaps, and the three-dimensional data is calculated respectively. At this time, in the present embodiment, a case where the range of the overlapping portion between the image of the first portion 2M and the image of the second portion 2N is set in advance will be described as an example. Note that the range of the overlapping portion is not limited to being set in advance, and may be set, for example, during imaging. In the present embodiment, the image of the first portion 2M and the image of the second portion 2N include three common feature regions (here, feature regions of the shape of the measurement object 2 as an example) For example, half of the two images are set as the overlapping range, but the present invention is not limited to this.

  FIG. 5 is a flowchart for explaining an example of the measurement method according to the first embodiment. FIG. 6 is a diagram illustrating an example of a positional relationship between the measurement object 2 and the shape measuring apparatus 1 in the measurement method. 7 to 10 are diagrams schematically showing images and the like displayed in the display area 70a in the measurement method along the processing order.

In the measurement method of the present embodiment, the user places the shape measuring device 1 at the measurement position A1 shown in FIG. 6 and captures an image of the first portion 2N of the measurement object 2. Thereafter, the user moves the shape measuring device 1 to the + Y side, places the shape measuring device 1 at the measurement position A2 shown in FIG. 6, and takes an image of the second portion 2N.
First, a case where the user measures the first portion 2M on the −Y side of the measurement object 2 will be described. In a state where the power of the shape measuring apparatus 1 is turned on, the control unit 62 causes the imaging unit 50 to capture a live view image of the measurement object 2 using natural light at a predetermined frame rate. The display control unit 67 displays the live view image Im1 captured by the imaging unit 50 in the display area 70a of the display device 70 as illustrated in FIG. From this state, the user moves the shape measuring apparatus 1 to the imaging position A1 and performs a shutter operation.

  When a shutter operation is performed by the user, the control unit 62 outputs a command signal to the light generation unit 20 and the scanning unit 40, and four types of stripe patterns P (see FIGS. 5A to 5D) in the first portion 2M. d) is projected, and the CCD camera 52a captures the measurement images of the first portion 2M on which the fringe patterns P are projected. And the calculating part 65 acquires the three-dimensional data (1st three-dimensional data) of the 1st part 2M based on four types of measurement images (step S01). This three-dimensional data is image data relating to the three-dimensional shape of the first portion 2M.

  Further, when the shutter operation is performed, the calculation unit 65 acquires the reference image ImR (see FIG. 8) of the first portion 2M by the imaging unit 50. As shown in FIG. 8, the reference image ImR is a still image of the first portion 2M. The computing unit 65 detects the feature region C at a plurality of positions in the acquired reference image ImR of the first portion 2M by image processing. As the feature region C, for example, three portions corresponding to the corners and concave portions of the measurement object 2 are detected. As shown in FIG. 7B, the display control unit 67 displays the live view image Im1 picked up by the image pickup unit 50 on the entire display area 70a (step 02) and at the right half of the acquired reference image ImR. (See FIG. 8) is displayed as an overlay image Im2 so as to be superimposed on the left half region of the live view image Im1 (step S03).

  In this embodiment, the next imaging is performed by moving the shape measuring apparatus 1 in the + Y direction. Therefore, the display control unit 67 arranges the overlay image Im2 on the side opposite to the moving direction of the shape measuring apparatus 1, that is, the left half region of the display region 70a. At this time, the display control unit 67 displays a portion corresponding to the right half (+ Y side) of the reference image ImR as the overlay image Im2.

  Further, the display control unit 67 displays the overlay image Im2 with a predetermined transparency. Therefore, the live view image Im1 is displayed so as to be transparent from the overlay image Im2 in the area where the overlay image Im2 is displayed.

  Next, as shown in FIG. 9A, the display control unit 67 displays an indicator 71 indicating the transparency of the overlay image Im2 in the display area 70a. In this case, the higher the transparency of the overlay image Im2, the greater the degree that the live view image Im1 can be seen through from the overlay image Im2. For example, when the transparency of the overlay image Im2 is the highest, the live view image Im1 is displayed in the same manner as when the overlay image Im2 is not superimposed. On the contrary, the lower the transparency of the overlay image Im2, the smaller the degree to which the live view image Im1 can be seen through from the overlay image Im2. For example, when the transparency of the overlay image Im2 is the lowest, the live view image Im1 is invisible. For example, the indicator 71 is set so that the transparency decreases as the gauge reaches the upper end (the degree to which the live view image Im1 is seen through increases), but is not limited thereto. After the indicator 71 is displayed, the user can check the live view image Im1 and the overlay image Im2 displayed in the display area 70a, and can adjust the transparency of the overlay image Im2. When the transparency of the overlay image Im2 is changed by the user, the display control unit 67 changes the display of the overlay image Im2 based on the changed transparency (step S04). In addition, the display control unit 67 changes the gauge of the indicator 71 according to the changed transparency. For example, FIG. 9B shows an example in which the user sets the overlay image Im2 to have higher transparency than the state of FIG. 9A. In the display state of FIG. 9B, the live view image Im1 displayed through the overlay image Im2 is clearer than the display state of FIG. 9A. Moreover, the gauge of the indicator 71 has moved downward. As described above, the user can easily identify the live view image Im1 and the overlay image Im2 in the display area 70a by changing the display state of the overlay image Im2.

  Next, the user measures the second portion 2N on the + Y side of the measurement object 2. When the user moves the shape measuring apparatus 1 to the + Y side (for example, the position A1a illustrated in FIG. 6), the live view image Im1 is switched as illustrated in FIG. At this time, the user compares the live view image Im1 displayed on the display area 70a with the overlay image Im2, and displays the measurement target 2 indicated by the live view image Im1 with respect to the display of the measurement target 2 indicated by the overlay image Im2. The shape measuring apparatus 1 is moved in the + Y direction so that the display of 2 overlaps. At this time, since the display state of the overlay image Im2 is changed and the live view image Im1 and the overlay image Im2 can be easily identified, the user can easily superimpose these images.

  Thereafter, as illustrated in FIG. 10B, the user adjusts the position of the shape measuring apparatus 1 to a position where the live view image Im1 and the overlay image Im2 overlap (for example, the imaging position A2 illustrated in FIG. 6). Perform the shutter operation. When this shutter operation is performed, the control unit 62 receives a signal indicating that the shutter operation has been performed from the operation unit 61. Further, when the shutter operation is performed, the projection optical system 30 and the light receiving optical system 51 may be focused by measuring the distance from the second portion 2N.

  When the shutter operation is performed, the control unit 62 outputs a command signal to the light generation unit 20 and the scanning unit 40, and four types of stripe patterns P are formed on the second portion 2N on the + Y side of the measurement object 2. The CCD camera 52a picks up a measurement image of the second portion 2N on which the fringe pattern P is projected. Thereafter, the calculation unit 65 acquires the three-dimensional data (second three-dimensional data) of the second portion 2N based on the four types of measurement images. This three-dimensional data is the shape data of the measurement object 2 shown in the live view image Im1 when the shutter operation is performed, for example, image data relating to the three-dimensional shape of the second portion 2N. Thus, the second three-dimensional data is data corresponding to the live view image Im1. The measurement image of the second three-dimensional data is captured in a state where the measurement object 2 displayed in the overlay image Im2 and the measurement object 2 displayed in the live view image Im1 are substantially overlapped. is there. Therefore, the second three-dimensional data has an overlapping portion with the first three-dimensional data. In this way, the arithmetic processing unit 60 acquires a plurality of three-dimensional data related to the three-dimensional shape of the measurement object 2 (step S05). The range of the overlapping portion between the acquired second three-dimensional data and the first three-dimensional data is almost a preset range.

  In addition, when the shutter operation is performed, the calculation unit 65 acquires a reference image of the second portion 2N by the imaging unit 50. The computing unit 65 detects feature regions at a plurality of positions in the acquired reference image of the second portion 2N. Then, the display control unit 67 displays a part of the acquired reference image of the second portion 2N as an overlay image so as to be superimposed on the live view image. Thereafter, when the user acquires the entire three-dimensional data of the measurement object 2, the user ends imaging. Further, when the three-dimensional data of a part of the measuring object 2 is not obtained, the above operation is repeated until three-dimensional data of all the parts of the measuring object 2 is obtained. That is, the newly displayed overlay image and the live view image are compared, the position of the shape measuring apparatus 1 is adjusted so that the live view image and the overlay image overlap, and then the shutter operation is performed.

  Next, the computing unit 65 connects the acquired three-dimensional data of the first part 2M and the three-dimensional data of the second part 2N so that the shape of the measurement object 2 is restored. When connecting these three-dimensional data, first, the rotation and translation from the shape measuring apparatus 1 to the first part 2M and the second part 2N are calculated. For example, the correspondence between the calculated three-dimensional data X1 of the first portion 2M and the two-dimensional coordinates of the feature region included in the acquired reference image ImR of the first portion 2M is transferred from the shape measuring device 1 to the first portion 2M. Obtain rotation R1 and translation t1. In addition, for example, the rotation from the shape measuring device 1 to the second portion 2N by the correspondence between the calculated three-dimensional data X2 of the second portion 2N and the two-dimensional coordinates of the feature region of the acquired reference image of the second portion 2N. R2 and translation t2 are calculated. The feature region is a region common between the reference image ImR of the first portion 2M and the reference image of the second portion 2N. In this case, rotation and translation can be calculated by using academic papers (eg, V. Lepetit et al. “EPnP: An Accurate O (n) Solution to the PnP Problem”, International Journal Of Computer Vision, vol. 81, p. 155-166, 2009.) and published publications can be used.

  Next, the computing unit 65 obtains the rotation Ra and the translation ta of the shape measuring apparatus 1 by the following [Equation 1] using the obtained R1, t1, R2, and t2. The rotations R1, R2, and Ra are represented by determinants, and the translations t1, t2, and ta are represented by vectors.

  As described above, in the shape measuring apparatus 1, the rotation and the translation change by Ra and ta, respectively, between the first position and the second position. Therefore, the three-dimensional data X2 acquired at the second position can be converted into the three-dimensional data when viewed from the first position by converting using the following equation [2].

  Thus, after converting the three-dimensional data X1 and X2, the calculation unit 65 connects the three-dimensional data X1 and the three-dimensional data X2 by superimposing the feature regions. And the calculating part 65 measures the shape of the measuring object 2 based on the connected three-dimensional data (step S06).

  Here, when three-dimensional data is connected as described above, the range of the overlapping portion between the image of the first portion 2M and the image of the second portion 2N is set to a preset range. is important. Conventionally, in the case where a marker whose light reflectance is significantly different from other regions is used as a feature region, a predetermined amount is obtained while checking the degree of overlap of the image of the second portion 2N with respect to the image of the first portion 2M. It was possible to set the range of the overlapping portion so as to include a number of common feature areas (for example, three markers). However, when a region other than the marker (change in the shape of the measurement object 2 or light reflectance) is used as the feature region without using the marker, a change between the feature region and another region (for example, contrast on the image) ) Is often small, and it is difficult to set the overlapping range so that a predetermined number of common feature regions are included. Therefore, it becomes difficult to capture images while confirming how much the image of the first part 2M and the image of the second part 2N overlap, and the range of the overlapping part is too wide with respect to the preset range. Sometimes it was too narrow. When the range of the overlapping portion is too wide, the number of imaging required for measuring the entire measurement object 2 increases. In addition, when the range of the overlapping portion is too narrow, since the desired number (for example, three) of feature regions are not included in the overlapping portion, the accuracy of the connection between the three-dimensional data is reduced, and the three-dimensional data is accurately obtained. It is necessary to redo the imaging of the measurement object 2 so that they can be connected to each other. For this reason, it takes time and effort to capture images, and it has been difficult to efficiently link three-dimensional data.

  The reason why the first part of the three-dimensional data and the second part of the three-dimensional data are connected using the feature region on the object to be measured without using the marker is as follows. There are cases where it is not possible to place a marker because of the large amount of time (or the time to remove the placed marker) and the measurement object.

  Therefore, in this embodiment, half of the reference image ImR when the measurement object 2 is imaged is displayed as an overlay image Im2 on the live view image Im1, and the overlap portion Im2 and the live view image Im1 are compared while comparing the overlay image Im2. Alignment is to be performed. Accordingly, since the overlay image Im2 and the live view image Im1 can be easily identified, it is easy to set the overlapping portion between the overlay image Im2 and the live view image Im1 to a preset range. However, when an overlay image is simply displayed, the overlay image and the live view image cannot be sufficiently identified depending on the shape or color of the measurement object 2, and it is difficult to determine whether or not the overlapping portion is appropriate. There is a case.

  In such a case, in this embodiment, the overlay image Im2 and the live view image Im1 can be easily identified by changing the transparency of the overlay image Im2. For this reason, it is possible to easily determine whether or not the overlapping portion between the overlay image Im2 and the live view image Im1 is appropriate.

  As described above, according to the first embodiment, the live view image Im1 of the measurement object 2 is displayed, and one of the images of the measurement object 2 captured when the first three-dimensional data is acquired. Corresponding to the live view image Im1 having a portion overlapping with the first three-dimensional data, displaying the image as an overlay image Im2 and overlapping the live view image Im1, changing the display state of the overlay image Im2 Since it includes connecting the second 3D data of the measurement object 2 and the first 3D data, the overlay image Im2 and the live view image Im1 can be easily identified, and the overlay image Im2 and the live view are displayed. It becomes easy to set the overlapping portion with the image Im1 within a preset range. Thereby, the frequency | count of imaging for measuring the whole measuring object 2 can be suppressed. In addition, since the three-dimensional data can be connected with high accuracy, the possibility that the measurement object 2 is imaged again is reduced. Therefore, the labor required for imaging can be reduced, and three-dimensional data can be efficiently connected.

Second Embodiment
Next, a second embodiment will be described. In the present embodiment, the case where the color of the overlay image is changed will be described as an example of the display state of the overlay image Im2. FIGS. 11A and 11B are diagrams illustrating an example of an image displayed in the display area 70a. Note that the color is represented by hue, brightness, and saturation.

  For example, when the user performs a shutter operation in a state where the first portion 2M of the measurement object 2 is displayed on the display area 70a, the control unit 62 determines the three-dimensional data (first three-dimensional data) of the first portion 2M. ). In addition, when the shutter operation is performed, the calculation unit 65 acquires a reference image of the first portion 2M by the imaging unit 50. The computing unit 65 detects feature regions at a plurality of positions in the acquired reference image of the first portion 2M. Further, as illustrated in FIG. 11A, the display control unit 67 displays a part of the acquired reference image as an overlay image Im2 so as to be superimposed on the live view image Im1. At this time, the display control unit 67 displays the overlay image Im2 in a state where the color is changed with respect to the image of the measurement object 2.

  About the color of overlay image Im2, the display control part 67 may set automatically, and the display control part 67 may set according to the value set by the user. In the present embodiment, the reference image, the live view image Im1, and the overlay image Im2 are monochrome images. For this reason, when the display control unit 67 sets the color of the overlay image Im2, for example, the overlay image Im2 may be displayed so as to invert the brightness of the portion displayed in the live view image Im1. When the user sets the color of the overlay image Im2, the display control unit 67 may perform control to display information representing the color on the display screen of the display device 70.

  For example, the user can confirm the live view image Im1 and the overlay image Im2 displayed in the display area 70a, and can change the color of the overlay image Im2. When the color of the overlay image Im2 is changed by the user, the display control unit 67 changes the display of the overlay image Im2 based on the changed color, as shown in FIG.

  Thus, according to the second embodiment, the overlay image Im2 and the live view image Im1 can be easily identified by changing the color of the overlay image Im2. For this reason, it is possible to easily determine whether or not the overlapping portion between the overlay image Im2 and the live view image Im1 is appropriate. Thereby, the connection of three-dimensional data can be performed accurately and efficiently.

FIG. 12 is a diagram illustrating another example of an image displayed in the display area 70a.
When the calculation unit 65 detects the feature region C from the reference image of the first portion 2M, the display control unit 67 changes only the color of the portion related to the feature region C in the reference image as shown in FIG. May be displayed. Note that the feature region C may be, for example, a plurality of (for example, three) markers arranged on the measurement object 2. As a result, the feature region C stands out, and it becomes easier to identify the overlay image Im2 (feature region C) and the live view image Im1.

<Third Embodiment>
Next, a second embodiment will be described. In the present embodiment, the case where the overlay image blinks will be described as an example of the display state of the overlay image Im2. FIG. 13A is a diagram illustrating an example of an image displayed in the display area 70a.

  As illustrated in FIG. 13A, the display control unit 67 displays a part of the acquired reference image as an overlay image Im2 so as to be superimposed on the live view image Im1. At this time, the display control unit 67 may display the overlay image Im2 in a blinking state as shown in FIG.

  Regarding blinking of the overlay image, the display control unit 67 can set a blinking interval and a blinking range. This setting may be automatically set by the display control unit 67 or may be set according to a value set by the user. When the user sets the blinking interval and blinking range of the overlay image, the display control unit 67 performs control to display information regarding the blinking interval and blinking range on the display screen of the display device 70.

  For example, the user can confirm the live view image Im1 and the overlay image Im2 displayed in the display area 70a, and can change the blinking state of the overlay image Im2. When the flashing state of the overlay image Im2 is changed by the user, the display control unit 67 changes the flashing state of the overlay image Im2 based on the changed mode.

  Thus, according to the third embodiment, the overlay image Im2 and the live view image Im1 can be easily identified by blinking the overlay image Im2. For this reason, it is possible to easily determine whether or not the overlapping portion between the overlay image Im2 and the live view image Im1 is appropriate. Thereby, the connection of three-dimensional data can be performed accurately and efficiently.

FIG. 13B is a diagram showing another example of an image displayed in the display area 70a.
When the calculation unit 65 detects the feature region C from the reference image of the first portion 2M, the display control unit 67 blinks only the portion related to the feature region C in the reference image as shown in FIG. May be displayed as follows. The feature region C may be a plurality of (for example, three) markers arranged on the measurement object 2. As described above, the display control unit 67 can change the blinking range. Further, the display control unit 67 may change the blinking state of only the portion related to the feature region C. As a result, the feature region C stands out, and it becomes easier to identify the overlay image Im2 (feature region C) and the live view image Im1.

<Modification>
Next, a modified example of the present invention will be described. The following modifications can be applied in appropriate combination with the configuration of the above-described embodiment.
For example, in each of the above embodiments, the case where the overlay image Im2 is displayed on the left half of the display area 70a has been described as an example, but the present invention is not limited to this. For example, depending on the shape of the measurement object 2, the shape measuring apparatus 1 may be moved in the vertical direction (Z direction) or the left direction (−Y direction) instead of the right direction (+ Y direction) as in the above embodiment. is there. In such a case, the overlay image Im2 is displayed on the opposite side of the display region 70a from the moving direction of the shape measuring apparatus 1. For example, when the shape measuring apparatus 1 is moved downward (for example, in the −Z direction), the overlay image Im2 is displayed on the upper half of the display area 70a. When the shape measuring apparatus 1 is moved upward (for example, in the + Z direction), the overlay image Im2 is displayed on the lower half of the display area 70a. When the shape measuring apparatus 1 is moved to the left side (for example, in the −Y direction), the overlay image Im2 is displayed on the right half of the display area 70a. In addition, when changing the moving direction of the shape measuring apparatus 1, the display position of the overlay image Im2 is changed according to the change of the moving direction of the shape measuring apparatus 1. When displaying the overlay image Im2 on the upper half of the display area 70a, the display control unit 67 displays the lower half of the reference image. When displaying the overlay image Im2 in the lower half of the display region 70a, the display control unit 67 displays the upper half of the reference image. When displaying the overlay image Im2 on the right half of the display area 70a, the display control unit 67 displays the left half of the reference image.

  Further, as shown in FIG. 14A, the display range of the overlay image Im2 can be changed by a user operation or the like, for example. In this case, the display range of the overlay image Im2 may be a range wider than a half of the display region 70a or a narrow range. When the display range of the overlay image Im2 is changed, the display control unit 67 translates the overlay image Im2 in the Y direction in the coordinate system of the live view image Im1. Thereby, since the display range of the overlay image Im2 can be changed according to the shape, color, etc. of the measurement object 2, the overlay image Im2 and the live view image Im1 can be easily identified. The same applies to the case where the overlay image Im2 is displayed on the upper side, lower side, and right side of the display area 70a. Further, for example, the display control unit 67 may automatically change the display range of the overlay image Im2 based on the image processing by the calculation unit 65. In this case, the display control unit 67 can change the display range as appropriate in accordance with the distribution of the feature region included in the overlay image Im2. At this time, the number of feature areas included in the overlay image Im2 may be four or more, and the number of feature areas may be two.

  As shown in FIG. 14B, in the first embodiment, the overlay image Im2 can be set to have a transparency of 0 (opaque). In this case, the portion of the live view image Im1 that overlaps the overlay image Im2 is not visible. When a linear side 2L extending in the Y direction (left-right direction in FIG. 14B) is provided in the measurement object 2, even if the transparency of the overlay image Im2 is 0, the live view image Im1 and the overlay image Im2 The position of the shape measuring apparatus 1 can be adjusted while observing how the side 2L overlaps between the two.

  In the first embodiment, the configuration in which only one imaging unit 50 is provided has been described as an example. However, the present invention is not limited to this. FIG. 15 is a diagram illustrating an example of a shape measuring apparatus 1A according to a modification.

  As illustrated in FIG. 15, the shape measuring apparatus 1 </ b> A includes a second imaging unit 150 that can be controlled independently of the imaging unit 50 in addition to the imaging unit 50. In addition, about another structure, for example, the structure of the projection part 10, the arithmetic processing part 60, the structure of the measuring object 2, etc., it is the same as 1st Embodiment. The second imaging unit 150 includes a light receiving optical system 151 and an imaging device 152. For example, a color camera is used as the second imaging unit 150. For this reason, the image imaged with the 2nd imaging part 150 turns into a color image.

  In such a configuration, the display control unit 67 overlays the images of a plurality of (for example, three) markers M among the images captured by the second imaging unit 150 as feature regions C, as shown in FIG. It may be displayed on the image Im2. In this case, the entire live view image Im1 and the portion other than the feature region C in the overlay image Im2 are displayed as a monochrome image. In addition, the feature region C of the overlay image Im2 is displayed as a color image. Therefore, the user can easily identify the overlay image Im2 and the live view image Im1. When this modification is combined with the second embodiment, the reference image, the live view image Im1, and the overlay image Im2 are color images. Therefore, when changing the color of the overlay image Im2, not only the lightness but also the hue and saturation may be changed.

  Note that a monochrome camera may be used as the second imaging unit 150 instead of a color camera. In addition, although the characteristic part C was demonstrated as an image of the marker M, it is not limited to this, For example, the partial image of the measuring object 2 may be sufficient.

<Structure manufacturing system and structure manufacturing method>
FIG. 17 is a block diagram illustrating an example of an embodiment of a structure manufacturing system. The structure manufacturing system SYS illustrated in FIG. 17 includes the shape measuring device 1 (or the shape measuring device 1A), the design device 710, the molding device 720, the control device (inspection device) 730, and the repair device 740. .

  The design apparatus 710 creates design information related to the shape of the structure. Then, the design device 710 transmits the produced design information to the molding device 720 and the control device 730. Here, the design information is information indicating the coordinates of each position of the structure. Further, the measurement object is a structure.

  The forming device 720 forms a structure based on the design information transmitted from the design device 710. The molding process of the molding apparatus 720 includes casting, forging, cutting, or the like. The shape measuring devices 1 and 1 </ b> A measure the three-dimensional shape of the structure (measurement object 2) produced by the forming device 720, that is, the coordinates of the structure. Then, the shape measuring devices 1 and 1 </ b> A transmit information indicating the measured coordinates (hereinafter referred to as shape information) to the control device 730.

  The control device 730 includes a coordinate storage unit 731 and an inspection unit 732. The coordinate storage unit 731 stores design information transmitted from the design device 710. The inspection unit 732 reads design information from the coordinate storage unit 731. Further, the inspection unit 732 compares the design information read from the coordinate storage unit 731 with the shape information transmitted from the shape measuring devices 1 and 1A. And the test | inspection part 732 test | inspects whether the structure was shape | molded according to design information based on the comparison result.

  Further, the inspection unit 732 determines whether or not the structure molded by the molding device 720 is a non-defective product. Whether or not the structure is a non-defective product is determined based on, for example, whether or not the error between the design information and the shape information is within a predetermined threshold range. If the structure is not molded according to the design information, the inspection unit 732 determines whether the structure can be repaired according to the design information. If it is determined that it can be repaired, the inspection unit 732 calculates a defective portion and a repair amount based on the comparison result. Then, the inspection unit 732 transmits information indicating a defective portion (hereinafter referred to as defective portion information) and information indicating a repair amount (hereinafter referred to as repair amount information) to the repair device 740.

  The repair device 740 processes the defective portion of the structure based on the defective portion information and the repair amount information transmitted from the control device 730.

  FIG. 18 is a flowchart showing processing by the structure manufacturing system SYS, and shows an example of an embodiment of a structure manufacturing method. As shown in FIG. 18, the design device 710 creates design information related to the shape of the structure (step S31). The design device 710 transmits the produced design information to the molding device 720 and the control device 730. The control device 730 receives the design information transmitted from the design device 710. Then, the control device 730 stores the received design information in the coordinate storage unit 731.

  Next, the molding apparatus 720 molds the structure based on the design information created by the design apparatus 710 (step S32). Then, the shape measuring devices 1 and 1A measure the three-dimensional shape of the structure formed by the forming device 720 (step S33). Thereafter, the shape measuring devices 1 and 1A transmit shape information, which is a measurement result of the structure, to the control device 730. Next, the inspection unit 732 compares the shape information transmitted from the shape measuring apparatuses 1 and 1A with the design information stored in the coordinate storage unit 731, and whether the structure has been molded according to the design information. Whether or not is checked (step S34).

  Next, the inspection unit 732 determines whether or not the structure is a good product (step S35). If it is determined that the structure is a non-defective product (step S35: YES), the process by the structure manufacturing system SYS is terminated. On the other hand, when the inspection unit 732 determines that the structure is not a non-defective product (step S35: NO), the inspection unit 732 determines whether the structure can be repaired (step S36).

  When the inspection unit 732 determines that the structure can be repaired (step S36: YES), the inspection unit 732 calculates the defective portion of the structure and the repair amount based on the comparison result of step S34. Then, the inspection unit 732 transmits the defective part information and the repair amount information to the repair device 740. The repair device 740 performs repair (rework) of the structure based on the defective part information and the repair amount information (step S37). Then, the process proceeds to step S33. That is, the process after step S33 is performed again with respect to the structure which the repair apparatus 740 performed repair. On the other hand, when the inspection unit 732 determines that the structure can be repaired (step S36: NO), the process by the structure manufacturing system SYS is terminated.

  As described above, in the structure manufacturing system SYS and the structure manufacturing method, based on the measurement result of the structure by the shape measuring apparatuses 1 and 1A, the inspection unit 732 determines whether the structure is manufactured according to the design information. judge. Accordingly, it can be accurately determined whether or not the structure manufactured by the molding apparatus 720 is a non-defective product, and the determination time can be shortened. Further, in the structure manufacturing system SYS described above, when the inspection unit 732 determines that the structure is not a non-defective product, the structure can be repaired immediately.

  In the structure manufacturing system SYS and the structure manufacturing method described above, the forming device 720 may be configured to execute the processing again instead of the repair device 740 executing the processing.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. Various modifications or improvements can be added to the above-described embodiment without departing from the spirit of the present invention. In addition, one or more of the requirements described in the above embodiments may be omitted. Such modifications, improvements, and omitted forms are also included in the technical scope of the present invention. In addition, the configurations of the above-described embodiments and modifications can be applied in appropriate combinations. In addition, as long as it is permitted by law, the disclosure of all published publications and US patents related to the X-ray apparatus and the like cited in the above embodiments and modifications are incorporated herein by reference.

  For example, in each of the above-described embodiments and modifications, the first direction D1 and the second direction D2 are orthogonal to each other, but are orthogonal if the first direction D1 and the second direction D2 are different directions. You don't have to. For example, the second direction D2 may be set to an angle of 60 degrees or 80 degrees with respect to the first direction D1.

  Further, in each of the above-described embodiments and modifications, each drawing shows one or more optical elements, but unless the number to be used is specified, it is used as long as the same optical performance is exhibited. The number of optical elements to be performed is arbitrary.

  In each of the above-described embodiments and modifications, the light for generating the structured light 101 by the light generation unit 20 or the like is light having a wavelength in the visible light region, light having a wavelength in the infrared region, or light having a wavelength in the ultraviolet region. Either of these may be used. By using light having a wavelength in the visible light region, the user can recognize the projection region 200. By using a red wavelength in the visible light region, damage to the measurement object 2 can be reduced. Further, in each of the above-described embodiments and modifications, the reference image has been described by taking, for example, an image of the measurement object 2 by natural light as an example, but the present invention is not limited to this, for example, by illumination light such as white light. It may be an image of the measuring object 2. The illumination light is not limited to white light and may be other types of light.

  Further, in each of the above-described embodiments and modifications, the scanning unit 40 uses an optical element that reflects structured light, but is not limited thereto. For example, a diffractive optical element, a refractive optical element, parallel flat glass, or the like may be used. The structured light may be scanned by vibrating a refractive optical element such as a lens with respect to the optical axis. As this refractive optical element, a part of the optical elements of the projection optical system 30 may be used.

  In each of the above-described embodiments and modifications, the CCD camera 52a is used as the imaging unit 50, but the present invention is not limited to this. For example, an image sensor such as a CMOS image sensor (CMOS: Complementary Metal Oxide Semiconductor) may be used instead of the CCD camera. In the above embodiment, the case where a monochrome camera is used as the imaging unit 50 has been described as an example. However, the present invention is not limited to this, and a color camera may be used.

  In each of the above-described embodiments and modifications, the 4-bucket method is used in which the phase of the fringe pattern P used in the phase shift method is shifted four times during one period. However, the present invention is not limited to this. For example, a 5-bucket method in which one period 2π of the phase of the fringe pattern P is divided into 5 or a 6-bucket method in which the period is also divided into 6 may be used.

  In each of the above-described embodiments and modifications, the phase shift method is used, but the three-dimensional shape of the measurement object 2 may be measured using the spatial code method.

  Moreover, in each above-mentioned embodiment and modification, although the fringe pattern P was represented by white and black, it is not limited to this, Either one or both may be a single color. For example, the stripe pattern P may be generated in white and red.

  Moreover, in each said embodiment and modification, the display control part 67 may display overlay image Im2 in the state which changed the display only of the outline part among the measuring objects 2. FIG.

  In the first embodiment, the case where the transparency of the overlay image Im2 is set by a user operation has been described as an example. However, the present invention is not limited to this and is automatically set by the display control unit 67. May be. In this case, the display control unit 67 may be configured to set the transparency according to the brightness of the live view image Im1. In the first embodiment, the range in which the transparency of the overlay image Im <b> 2 can be changed may be changed by a user operation or control by the display control unit 67. In the first embodiment, not only the transparency of the overlay image Im2 but also the transparency of the live view image Im1 may be changed, or the transparency of both the overlay image Im2 and the live view image Im1 may be changed. .

  In the first embodiment, the feature region C is detected from the reference image ImR, and the range of the overlay image Im2 is set based on the position of the feature region C and the like. However, the present invention is not limited to this. The range of the overlay image Im2 may be set with the measurement object 2 as a reference. For example, an area including a predetermined part of the measurement object 2 in the reference image ImR may be set to be the overlay image Im2. In this case, the user only needs to take an image so as to include a predetermined portion of the measuring object 2 while confirming the display area 70a at the time of imaging.

  In the above embodiment, the case where the feature region C is detected by the calculation unit 65 has been described as an example. However, the present invention is not limited to this. For example, the user designates the feature region on the touch panel. Also good.

  Moreover, in the said embodiment or modification, although the case where a part on the measuring object 2 was used as the feature region C was described as an example, a marker may be used as the feature region in the above description. Even in this case, since the degree of overlap between the image of the first part 2M and the image of the second part 2N is easier to confirm than in the past, the overlapping part of the two images is set in a preset range. Becomes easy. Therefore, the possibility that the measurement object 2 is imaged again becomes lower and the labor required for imaging can be reduced, so that the three-dimensional data can be efficiently connected.

  In the second embodiment, the case where the color is automatically set by the display control unit 67 has been described as an example. However, the present invention is not limited to this, and the user can set the color. Good. In this case, the display control unit 67 displays, for example, an operation area for the user to set a color on the touch panel on the display area 70a. As such an operation area, for example, an indicator that can individually change the hue, lightness, and saturation may be displayed, or a plurality of colors may be displayed side by side so that the user can directly select a color. Also, the hue, lightness, and saturation may be changed by inputting numerical values.

  Note that all functions of the arithmetic processing unit 60 may not be housed in a portable case, and some functions of the arithmetic processing unit 60 (the arithmetic unit, the image storage unit, the display control unit, and the setting information storage) May be provided to an external computer. In addition, the present invention is not limited to the portable shape measuring device 1, for example, a measuring machine provided with a three-dimensional measuring unit on an articulated arm, or a three-dimensional measuring unit on a stage on which a measurement object 2 is placed. The present invention can also be applied to a stationary shape measuring apparatus such as a measuring machine configured to be movable.

  Even in this case, similarly to the above-described embodiment, it is not necessary to synchronize the reciprocal vibration of the MEMS mirror and the light intensity emitted from the laser diode, and complicated and sophisticated synchronization control is not necessary. When carrying the shape measuring device housed in a portable case where the projection unit 10, the imaging unit 50, the arithmetic processing unit 60, and the display device 70 are carried, the external measurement environment (temperature, humidity, atmospheric pressure, etc.) is likely to change. However, even if the external environment changes, the shape of the measurement object 2 can be measured with high accuracy.

  Moreover, you may implement | achieve the one part structure of the shape measuring apparatus 1 with a computer. For example, the calculation unit processing unit 60 may be realized by a computer. In this case, the computer displays the live view image of the measurement object 2 and the image of the measurement object 2 captured when acquiring the three-dimensional data (first three-dimensional data) of the measurement object 2. A process for displaying a part of the image as an overlay image superimposed on the live view image, a process for changing the display state of the overlay image, and a measurement target corresponding to the live view image having a part overlapping the first three-dimensional data A process of executing the process of connecting the second three-dimensional data of the object 2 and the first three-dimensional data is executed.

  Note that the requirements of the above-described embodiments can be combined as appropriate. Some components may not be used. In addition, as long as it is permitted by law, the disclosure of the text is incorporated with the disclosure of all published publications and US patents related to the detection methods, shape measurement methods, shape measurement devices, etc. cited in the above embodiments and modifications. Part.

Im1 ... Live view image Im2 ... Overlay image C ... Characteristic region SYS ... Structure manufacturing system 1, 1A ... Shape measuring device 2 ... Measurement object 2M ... First part 2N ... Second part 50 ... Imaging part 60 ... Calculation processing part 62 ... Control unit 63 ... Setting information storage unit 65 ... Calculation unit 66 ... Image storage unit 67 ... Display control unit 70 ... Display device 70a ... Display area 71 ... Indicator 90 ... Housing 150 ... Second imaging unit 710 ... Design device 720 ... Molding device 730 ... Control device 740 ... Repair device

Claims (22)

A method for connecting three-dimensional data for connecting a plurality of three-dimensional data acquired for a measurement object,
Displaying a live view image of the measurement object;
Displaying a part of an image of the measurement object imaged when acquiring the first three-dimensional data as an overlay image superimposed on the live view image;
Changing the display state of the overlay image;
Connecting the first three-dimensional data with the second three-dimensional data of the measurement object corresponding to the live view image and having a portion overlapping with the first three-dimensional data. How to concatenate original data. The method of connecting three-dimensional data according to claim 1, wherein changing the display state of the overlay image includes changing the transparency of the overlay image. The three-dimensional data connection method according to claim 2, wherein the change of the transparency includes displaying information indicating the transparency on the display unit. The method of connecting three-dimensional data according to any one of claims 1 to 3, wherein changing the display state of the overlay image includes changing a color of the overlay image. The change of the display state of the overlay image includes blinking the overlay image or changing the blinking state of the overlay image. The three-dimensional data according to any one of claims 1 to 4, Consolidation method. The three-dimensional data connection method according to claim 5, wherein the change of the blinking state includes changing at least one of the blinking interval and range. The method for connecting three-dimensional data according to any one of claims 1 to 6, wherein changing the display state of the overlay image includes changing a display state of a part of the overlay image. The method for connecting three-dimensional data according to claim 7, wherein a part of the overlay image is a feature region included in the overlay image. A measurement method for measuring a three-dimensional shape of a measurement object,
Obtaining a plurality of three-dimensional data about the measurement object;
Connecting a plurality of three-dimensional data using the three-dimensional data connection method according to any one of claims 1 to 8,
Calculating a three-dimensional shape of the measurement object based on the connected result. A measuring device for measuring a three-dimensional shape of a measurement object,
An imaging unit for imaging the measurement object;
A display unit for displaying an image captured by the imaging unit;
An operation of causing the imaging unit to capture a live view image of the measurement object and displaying the live view image on the display unit, and obtaining the first three-dimensional data of the measurement object The image capturing unit captures a part of the image and the measurement target object part of the image is superimposed on the live view image as an overlay image and displayed on the display unit, and the display state of the overlay image is changed. Connecting the first three-dimensional data and the second three-dimensional data of the measurement object corresponding to the live view image having a portion overlapping with the first three-dimensional data And a control unit for performing
A measurement device comprising: a calculation unit that calculates a three-dimensional shape of the measurement object based on the connected result. The measuring apparatus according to claim 10, wherein the operation of changing the display state of the overlay image includes an operation of changing the transparency of the overlay image. The measurement apparatus according to claim 11, wherein the operation of changing the transparency includes displaying information indicating the transparency on the display unit. The measurement apparatus according to any one of claims 10 to 12, wherein the operation of changing the display state of the overlay image includes an operation of changing a color of the overlay image. The measurement device according to any one of claims 10 to 13, wherein the operation of changing the display state of the overlay image includes an operation of blinking the overlay image or changing the blinking state. The measurement device according to claim 14, wherein the operation of changing the blinking state includes an operation of changing at least one of the interval and range of the blinking. The measurement apparatus according to any one of claims 10 to 15, wherein the operation of changing the display state of the overlay image includes an operation of changing a display state of a part of the overlay image. The measurement apparatus according to claim 16, wherein a part of the overlay image is a characteristic region included in the overlay image. A second imaging unit that is provided separately from the imaging unit and captures an image of the measurement object;
18. The control unit displays at least a part of an image captured by the second imaging unit in an operation of changing a display state of the overlay image. 18. Measuring device. The second imaging unit is a color camera;
The measurement apparatus according to claim 18, wherein the image captured by the second imaging unit includes a color image. Creating design information on the shape of the structure;
Producing the structure based on the design information;
The shape measuring method according to claim 9, wherein the shape of the manufactured structure is measured;
Comparing the shape information on the shape of the structure obtained by the shape measuring method with the design information;
A structure manufacturing method comprising: A design device for creating design information on the shape of the structure;
A molding apparatus for producing the structure based on the design information;
The shape measuring device according to any one of claims 10 to 18, which measures the shape of the manufactured structure,
An inspection device for comparing shape information on the shape of the structure obtained by the shape measuring device with the design information;
Structure manufacturing system including. In the computer included in the measuring device that measures the three-dimensional shape of the measurement object,
Processing to display a live view image of the measurement object;
A process of displaying a part of an image of the measurement object imaged when acquiring the first three-dimensional data as an overlay image on the live view image;
A process of changing the display state of the overlay image;
A process of connecting the first three-dimensional data with the second three-dimensional data of the measurement object corresponding to the live view image and having a portion overlapping with the first three-dimensional data. Shape measurement program.