JP2025047138A - Three-dimensional measurement device and three-dimensional measurement method - Google Patents

Abstract

[Problem] To enable proper recognition of each recognition target using the same measurement device, even when a single component contains recognition targets with high and low optical reflectance. [Solution] A three-dimensional measurement device includes a projection device that projects patterned light with different periods onto a component having a first recognition target, an electrode, and a second recognition target with a lower optical reflectance than the electrode, an imaging device that separately images the first recognition target and the second recognition target onto which the patterned light is projected, and a processing device that performs image recognition of the first recognition target based on first image data of the first recognition target and image recognition of the second recognition target based on second image data of the second recognition target, and the measurement conditions, including the imaging conditions and the image recognition conditions, differ for the first recognition target and the second recognition target. [Selected Figure] Figure 2

Description

This disclosure relates to a three-dimensional measurement device and a three-dimensional measurement method.

The phase shift method disclosed in Patent Document 1 is known as a method for measuring three-dimensional objects. When measuring the three-dimensional shape of an object based on the phase shift method, a stripe pattern (pattern light) with a sinusoidal brightness distribution is projected onto the object while undergoing a phase shift. The object onto which the stripe pattern is projected is imaged, and an image of the object is obtained.

JP 2001-124534 A

One application of 3D measurement methods is the mounting of components on electronic devices. Recognition targets, such as component electrodes, are measured in 3D, and the components are mounted on a circuit board or the like based on the measurement results. Component electrodes have a high light reflectance and are easy to recognize, but are prone to saturation (saturation of image brightness values) due to excessive light reflection. On the other hand, component resin parts have a lower light reflectance than electrodes, and are difficult to recognize because they do not have as much contrast. Therefore, when a single component contains recognition targets with high light reflectance and those with low light reflectance, it is necessary to provide separate dedicated measurement devices suitable for recognizing each.

The objective of this aspect of the present invention is to make it possible to properly recognize each recognition target using the same measurement device, even when a single part contains recognition targets with high and low optical reflectance.

According to a first aspect of the present invention, there is provided a three-dimensional measuring device comprising: a projection device that projects pattern light with different phases onto a part having a first recognition target and a second recognition target having a lower light reflectance than the first recognition target; an imaging device that separately images the first recognition target and the second recognition target onto which the pattern light is projected; and a processing device that performs image recognition of the first recognition target based on first image data of the first recognition target and image recognition of the second recognition target based on second image data of the second recognition target, in which measurement conditions including imaging conditions and image recognition conditions are different for the first recognition target and the second recognition target.

According to a second aspect of the present invention, there is provided a three-dimensional measurement method comprising the steps of: projecting pattern light having different phases onto a part having a first recognition target and a second recognition target having a lower light reflectance than the first recognition target, and taking a first image of the first recognition target onto which the pattern light is projected; projecting the pattern light onto the part, and taking a second image of the second recognition target onto which the pattern light is projected; and performing image recognition of the first recognition target based on first image data of the first recognition target, and performing image recognition of the second recognition target based on second image data of the second recognition target, in which measurement conditions including imaging conditions and image recognition conditions are different for the first recognition target and the second recognition target.

According to this aspect of the present invention, even if a single part contains recognition targets with high light reflectance and recognition targets with low light reflectance, each recognition target can be properly recognized using the same measurement device.

FIG. 1 is a diagram illustrating an example of an electronic component mounting system according to an embodiment. FIG. 2 is a schematic diagram illustrating an example of a three-dimensional measuring apparatus according to an embodiment. FIG. 3 is a schematic diagram illustrating an example of a three-dimensional measuring apparatus according to an embodiment. FIG. 4 is a functional block diagram illustrating an example of a processing device according to the embodiment. FIG. 5 is a schematic diagram showing an example of pattern data according to the present embodiment. FIG. 6A is a side view and FIG. 6B is a bottom view showing an example of a component according to the embodiment. FIG. 7 is an explanatory diagram for explaining imaging of a component according to the embodiment. FIG. 8 is a diagram showing a specific example of the measurement conditions according to the embodiment. FIG. 9 is a schematic diagram for explaining the image synthesis process according to the embodiment. FIG. 10 is an explanatory diagram for explaining a recognition algorithm of the image recognition process executed by the processing device of the embodiment. FIG. 11 is an explanatory diagram illustrating an example of the first complementation process of the image recognition by the processing device of the embodiment. FIG. 12 is an explanatory diagram illustrating an example of the second complementation process of the image recognition by the processing device of the embodiment. FIG. 13 is a flowchart showing a three-dimensional measurement method according to the embodiment. FIG. 14 is a flowchart illustrating the operation of the electronic component mounting system according to the embodiment.

The following describes an embodiment of the present invention with reference to the drawings, but the present invention is not limited to this. The components of the embodiments described below can be combined as appropriate. Also, some components may not be used.

In the following explanation, an XYZ Cartesian coordinate system is set, and the positional relationship of each part is explained with reference to this XYZ Cartesian coordinate system. The direction parallel to the X-axis of a specified plane is defined as the X-axis direction, the direction parallel to the Y-axis perpendicular to the X-axis on the specified plane is defined as the Y-axis direction, and the direction parallel to the Z-axis perpendicular to the specified plane is defined as the Z-axis direction. The direction of rotation about the Z-axis is defined as the θZ direction. In the following explanation, the specified plane is appropriately referred to as the XY plane. In this embodiment, the XY plane and the horizontal plane are parallel to each other.

[Embodiment]
<Electronic component mounting system>
An embodiment will now be described. Fig. 1 is a diagram that illustrates an example of an electronic component mounting system 100 according to an embodiment. Fig. 1 is a diagram of the electronic component mounting system 100 as viewed from the -Y direction.

As shown in FIG. 1, the electronic component mounting system 100 includes a three-dimensional measuring device 1 and an electronic component mounting device 2. The electronic component mounting system 100 uses the three-dimensional measuring device 1 to three-dimensionally measure the shape of a component C, and based on the measurement results, the electronic component mounting device 2 mounts the component C on a substrate P. Below, the electronic component mounting device 2 will be described first, and then the three-dimensional measuring device 1 according to this embodiment will be described.

<Electronic component mounting equipment>
The electronic component mounting device 2 mounts the component C on the substrate P. The electronic component mounting device 2 includes a mounting head 22 that holds the nozzle 11 that holds the component C, an electronic component supply device 23 that supplies the component C, a substrate transport device 24 that transports the substrate P on which the component C is to be mounted, and a control device 29 that controls the electronic component mounting device 2.

The electronic component supply device 23 has, for example, a movement mechanism that moves a tray holding multiple components C in a horizontal direction. The electronic component supply device 23 supplies the components C to a supply position SP. The electronic component supply device 23 may be a component supply device of a tape supply type that supplies the components C held on a tape. In that case, the electronic component supply device 23 includes a tape reel that supports the tape, and a feeder that transports the components C held on the tape unwound from the tape reel.

The substrate transport device 24 has a conveyor belt that transports the substrate P and a clamping mechanism that fixes the position of the substrate P transported to the mounting position MP.

The mounting head 22 mounts the component C held by the nozzle 11 onto the substrate P. The mounting head 22 is moved in the X-axis direction and the Y-axis direction by the head driving device 25. The nozzle 11 is moved in the Z-axis direction and the θZ direction relative to the mounting head 22 by the nozzle driving device 26 provided on the mounting head 22. The nozzle 11 can be moved in four directions, the X-axis direction, the Y-axis direction, the Z-axis direction, and the θZ direction, by the head driving device 25 and the nozzle driving device 26. The mounting head 22 can be moved between a supply position SP defined by the electronic component supply device 23 and a mounting position MP defined on the substrate P.

The mounting head 22 holds the component C supplied from the electronic component supply device 23 with the nozzle 11 at the supply position SP, and then transports it to the board P placed at the mounting position MP. The mounting head 22 mounts the component C held by the nozzle 11 onto the board P placed at the mounting position MP.

The nozzle 11 removably holds the part C. The nozzle 11 may be a suction nozzle that suctions and holds the part C, or a gripping nozzle that clamps and holds the part C.

After the mounting head 22 holds the component C with the nozzle 11 at the supply position SP, it places the component C at the measurement side position TP of the three-dimensional measuring device 1 before transporting it to the substrate P. The measurement side position TP of the three-dimensional measuring device 1 refers to a position where the pattern light PL (see Figure 2) of the three-dimensional measuring device 1 can be irradiated.

The substrate transport device 24 is provided with a position measuring device 27 that measures the position of the through-hole formed in the substrate P placed at the mounting position MP. Measurement data from the position measuring device 27 indicating the position of the through-hole formed in the substrate P is output to the control device 29.

The control device 29 is connected to the head driving device 25, the nozzle driving device 26, the position measuring device 27, and the processing device 5 of the three-dimensional measuring device 1 described below. Before mounting the component C on the substrate P, the control device 29 measures the three-dimensional shape of the component C using the three-dimensional measuring device 1. Based on the measurement results of the component C by the three-dimensional measuring device 1, the mounting head 22 adjusts the relative position between the component C and the substrate P, as described below, and mounts the component C on the substrate P.

<3D measuring device>
As shown in Fig. 1, the three-dimensional measuring apparatus 1 includes a projection device 3 that projects pattern light PL onto a part C, an imaging device 4 that images the part C onto which the pattern light PL is projected, and a processing device 5 that performs image recognition based on the captured image data. In the example of Fig. 1, the three-dimensional measuring apparatus 1 is disposed below (in the -Z direction) the measurement position TP. The three-dimensional measuring apparatus 1 is installed with the imaging direction facing upward (in the +Z direction) so as to image the bottom surface of the part C held by the nozzle 11.

The three-dimensional measuring device 1 is positioned on the movement path of the mounting head 22 between the supply position SP and the mounting position MP at a position where it can irradiate the pattern light PL onto the component C held by the nozzle 11 and placed at the measurement position TP.

Figure 2 is a schematic diagram showing an example of a three-dimensional measuring device 1 according to an embodiment. Figure 3 is a schematic diagram showing an example of a three-dimensional measuring device 1 according to an embodiment. Figure 2 is a diagram of the three-dimensional measuring device 1, which is a component of the three-dimensional measuring device 1, viewed from the same direction as in Figure 1. Figure 3 is a diagram of the three-dimensional measuring device 1, which is a component of the three-dimensional measuring device 1, viewed from the vertically upward side, i.e., from the +Z direction.

In this embodiment, the part C has a first recognition target R1 and a second recognition target R2 that has a lower light reflectance than the first recognition target R1. The three-dimensional measuring device 1 performs three-dimensional measurement of both the first recognition target R1 and the second recognition target R2. Details of the part C, the first recognition target R1, and the second recognition target R2 will be described later.

As shown in FIG. 2, the projection device 3 projects pattern light PL with different phases onto the part C. The projection device 3 has a light source 31 that generates light, a light modulation element 32 that optically modulates the light generated from the light source 31 to generate the pattern light PL, and an emission optical system 33 that emits the pattern light PL generated by the light modulation element 32.

The light modulation element 32 includes a digital mirror device (DMD). The light modulation element 32 may include a transmissive liquid crystal panel or a reflective liquid crystal panel. The light modulation element 32 generates the pattern light PL based on the pattern data output from the processing device 5. The projection device 3 emits the pattern light PL that is patterned based on the pattern data.

The projection device 3 projects the pattern light PL onto the part C from multiple directions separately. The projection device 3 is equipped with a reflecting member 6 that changes the projection direction by reflecting the emitted light. The reflecting member 6 reflects the pattern light PL emitted from the projection device 3 and irradiates it onto the part C. The reflecting surface of the reflecting member 6 is flat.

The imaging device 4 has an imaging optical system 41 that forms an image of the pattern light PL reflected by the component C, and an imaging element 42 that acquires image data of the component C via the imaging optical system 41. The imaging element 42 is a solid-state imaging element including a CMOS image sensor (Complementary Metal Oxide Semiconductor Image Sensor) or a CCD image sensor (Charge Coupled Device Image Sensor).

In this embodiment, the imaging optical system 41 is a telecentric optical system. A telecentric optical system is an optical system in which the chief ray is parallel to the optical axis AX. Therefore, the light (chief ray) that enters the imaging field of view of the imaging device 4 and is imaged can be considered to be parallel to the optical axis AX.

In the embodiment, the imaging device 4 separately captures images of the first recognition target R1 and the second recognition target R2 onto which the pattern light PL is projected. The imaging device 4 generates first image data 61 (see FIG. 7) by capturing an image of the first recognition target R1 onto which the pattern light PL is projected, through the first imaging. The imaging device 4 generates second image data 62 (see FIG. 7) by capturing an image of the second recognition target R2 onto which the pattern light PL is projected, through the second imaging.

The processing device 5 includes a computer system and controls the projection device 3 and the imaging device 4. The processing device 5 has an arithmetic processing device including a processor such as a CPU (Central Processing Unit), a storage device including memory and storage such as a ROM (Read Only Memory) or a RAM (Random Access Memory), and an input/output interface including an input/output circuit capable of inputting and outputting signals and data. The arithmetic processing device performs arithmetic processing according to a computer program stored in the storage device.

In the embodiment, the processing device 5 performs image recognition of the first recognition target R1 based on the first image data 61 of the first recognition target R1, and performs image recognition of the second recognition target R2 based on the second image data 62 of the second recognition target R2. Note that in this specification, image recognition means acquiring at least one of the position and shape of the recognition target by three-dimensional measurement. Performing image recognition includes performing three-dimensional measurement.

The three-dimensional measuring device 1 uses a pattern projection method to measure the three-dimensional shape of the recognition targets (first recognition target R1, second recognition target R2) of the part C held by the nozzle 11 and positioned at the measurement position TP. The projection device 3 irradiates the part C with pattern light PL, for example, a stripe pattern light with a sinusoidal brightness distribution, while shifting the phase. As described above, the pattern light PL is irradiated onto the part C from multiple directions.

The imaging device 4 acquires image data of the part C onto which the pattern light PL is projected. The imaging device 4 acquires at least image data of the part C onto which the first pattern light PL1 is projected and image data of the part C onto which the second pattern light PL2 is projected. The direction of incidence of the first pattern light PL1 and the second pattern light PL2 incident on the part C are different.

As shown in FIG. 2, the first pattern light PL1 is pattern light PL emitted from the projection device 3 and directly irradiated onto the part C. The second pattern light PL2 is pattern light PL emitted from the projection device 3, reflected by the reflecting member 6, and irradiated onto the part C.

The processing device 5 controls the light modulation element 32 so as to change from one of a first irradiation state in which the first pattern light PL1 from the projection device 3 is irradiated onto the part C to a second irradiation state in which the second pattern light PL2 from the reflecting member 6 is irradiated onto the part C, to the other.

When irradiating the part C with the first pattern light PL1, the processing device 5 controls the light modulation element 32 so that the pattern light PL is emitted from the first region 331 of the emission surface 33S of the emission optical system 33 and is not emitted from the second region 332 of the emission surface 33S. The pattern light PL emitted from the first region 331 is irradiated directly to the part C as the first pattern light PL1 without passing through the reflecting member 6.

When irradiating the part C with the second pattern light PL2, the processing device 5 controls the light modulation element 32 so that the pattern light PL is emitted from the second region 332 of the emission surface 33S of the emission optical system 33 and is not emitted from the first region 331 of the emission surface 33S. The pattern light PL emitted from the second region 332 passes through the optical axis AX and is irradiated onto the reflecting member 6. The pattern light PL emitted from the second region 332 is irradiated onto the part C as the second pattern light PL2 via the reflecting member 6.

In the example shown in FIG. 1 and FIG. 2, the first region 331 is half the area of the emission surface 33S on the +X side of the optical axis of the emission optical system 33, and the second region 332 is half the area of the emission surface 33S on the -X side of the optical axis of the emission optical system 33. The pattern light PL emitted from the first region 331 is irradiated onto the component C that is held by the nozzle 11 and placed at the measurement position TP. The pattern light PL emitted from the second region 332 is irradiated onto the reflecting surface of the reflecting member 6, reflected by the reflecting surface of the reflecting member 6, and then irradiated onto the component C that is held by the nozzle 11 and placed at the measurement position TP.

As shown in Figures 1 and 2, the projection device 3, the imaging device 4, and the reflecting member 6 are supported by a housing 7. The relative positions of the projection device 3, the imaging device 4, and the reflecting member 6 are fixed by the housing 7. The incident surface 41S of the imaging device 4 can face a component C placed at the measurement position TP.

The reflecting member 6 is disposed at least partially around the optical axis AX of the imaging optical system 41 between the incident surface 41S and the part C disposed at the measurement position TP. The projection device 3 is disposed at least partially around the optical axis AX of the imaging optical system 41 between the incident surface 41S and the part C disposed at the measurement position TP. The reflecting member 6 is disposed at a position closer to the measurement position TP than the projection device 3.

As shown in FIG. 3, two projection devices 3 are arranged around the optical axis AX. In addition, two reflection members 6 are arranged around the optical axis AX.

The first reflecting member 6 is disposed on the -X side of the optical axis AX. The first projection device 3 is disposed on the +X side of the optical axis AX. The first projection device 3 can irradiate the first reflecting member 6 with the second pattern light PL2. The second pattern light PL2 reflected by the first reflecting member 6 is irradiated onto the component C placed at the measurement position TP. The first projection device 3 can also irradiate the first pattern light PL1 directly onto the component C placed at the measurement position TP.

The second reflecting member 6 is disposed on the +Y side of the optical axis AX. The second projection device 3 is disposed on the -Y side of the optical axis AX. The second projection device 3 is capable of irradiating the second reflecting member 6 with the second pattern light PL2. The second pattern light PL2 reflected by the second reflecting member 6 is irradiated onto the component C placed at the measurement position TP. In addition, the second projection device 3 is capable of irradiating the first pattern light PL1 directly onto the component C placed at the measurement position TP.

In this way, the first and second projection devices 3 and the first and second reflecting members 6 can project the pattern light PL from four directions onto the part C placed at the measurement position TP. The imaging device 4 acquires image data of the part C onto which the first pattern light PL1 from the first projection device 3 is projected, image data of the part C onto which the second pattern light PL2 from the first reflecting member 6 is projected, image data of the part C onto which the first pattern light PL1 from the second projection device 3 is projected, and image data of the part C onto which the second pattern light PL2 from the second reflecting member 6 is projected.

As described above, in this embodiment, there are two pairs of projection devices 3 and reflecting members 6, one pair facing each other in the X-axis direction and one pair facing each other in the Y-axis direction, but the present invention is not limited to this and may have one pair, or three or more pairs. In addition, the three-dimensional measuring device 1 may have multiple projection devices 3 provided independently (without reflecting members 6) for each projection direction.

In this embodiment, a three-dimensional measuring device 1 using a phase shift method including a projection device 3 consisting of a projection device 3 and a reflecting member 6, and an imaging device 4 is used as a preferred embodiment, but the present invention is not limited to this, and a form using an imaging image analysis method may also be used.

<Control device>
4 is a functional block diagram showing an example of the processing device 5 according to the embodiment. The processing device 5 includes an input/output unit 51, a pattern generating unit 52, an image data acquiring unit 53, a phase value calculating unit 54, and a three-dimensional shape calculating unit 55. The processing device 5 also includes a storage unit 56.

The input/output unit 51 is a functional unit that is realized by the input/output interface of the processing device 5 executing a three-dimensional measurement program for the three-dimensional measurement device 1 according to the embodiment to execute the three-dimensional measurement method according to the embodiment.

The pattern generation unit 52, image data acquisition unit 53, phase value calculation unit 54, and three-dimensional shape calculation unit 55 are functional units that are realized by the arithmetic processing unit of the processing device 5 executing a three-dimensional measurement program for the three-dimensional measurement device 1 according to the embodiment to execute the three-dimensional measurement method according to the embodiment. The arithmetic processing unit has a processor such as a CPU (Central Processing Unit) and a main memory including a non-volatile memory such as a ROM (Read Only Memory) and a volatile memory such as a RAM (Random Access Memory).

The storage unit 56 includes a non-volatile storage device such as a semiconductor memory. The storage unit 56 stores programs executed by the input/output unit 51, the pattern generation unit 52, the image data acquisition unit 53, the phase value calculation unit 54, and the three-dimensional shape calculation unit 55. The storage unit 56 stores part data, which is design information for part C. The part data records information such as the shape and dimensions of part C.

The pattern generation unit 52 generates pattern data. The pattern data generated by the pattern generation unit 52 is output to the light modulation element 32 via the input/output unit 51. The light modulation element 32 generates pattern light PL based on the pattern data generated by the pattern generation unit 52. The pattern data generated by the pattern generation unit 52 includes first pattern data for irradiating the part C with the first pattern light PL1 from the projection device 3 without passing through the reflecting member 6, and second pattern data for irradiating the part C with the second pattern light PL2 from the projection device 3 via the reflecting member 6.

Figure 5 is a schematic diagram showing an example of pattern data according to this embodiment. In Figure 5, areas that do not emit light are shown in black, and areas that emit light are shown in white. As shown in Figure 5 (A), when the first pattern light PL1 is irradiated onto the part C, the pattern generation unit 52 generates first pattern data and controls the light modulation element 32 so that the pattern light PL is emitted from the first area 331 of the emission surface 33S of the emission optical system 33 and the pattern light PL is not emitted from the second area 332 of the emission surface 33S.

As shown in FIG. 5(B), when the second pattern light PL2 is irradiated onto the part C, the pattern generation unit 52 generates second pattern data and controls the light modulation element 32 so that the pattern light PL is emitted from the second region 332 of the emission surface 33S of the emission optical system 33 and the pattern light PL is not emitted from the first region 331 of the emission surface 33S.

The image data acquisition unit 53 acquires image data from the image sensor 42 via the input/output unit 51. The image data acquisition unit 53 acquires image data of the part C onto which the first pattern light PL1 is projected by the first and second projection devices 3, and image data of the part C onto which the second pattern light PL2 is projected from the first and second reflecting members 6.

The phase value calculation unit 54 calculates the phase value of each of the multiple pixels of the image data based on the luminance (pixel value) of the image data. The phase value calculation unit 54 calculates the phase value of the pixel of the image data corresponding to the same point based on the luminance of the same point of the multiple image data of the part C onto which each of the phase-shifted pattern lights PL is projected. The phase value calculation unit 54 calculates the phase value of each of the multiple pixels of the image data based on the luminance of the multiple points of the image data.

The three-dimensional shape calculation unit 55 calculates the position data of each of the multiple points of part C corresponding to each of the multiple pixels of the image data based on the phase values of each of the multiple pixels of the image data, and calculates the three-dimensional shape of part C.

More specifically, the three-dimensional shape calculation unit 55 calculates the height (protruding length) of the first recognition target R1 and the second recognition target R2 based on the position data at the tip of the part C, and calculates the part height, which is the overall length of the part C in the vertical direction, as well as the height data of the first recognition target R1 and the height data of the second recognition target R2.

<Parts>
6A and 6B are a side view and a bottom view showing an example of a component C according to an embodiment. In the example, the component C is a connector component to be mounted on a substrate P. The component C has a body C1 and electrodes C2 extending from the body C1. The body C1 has a boss C3 for positioning and a screw hole C4 for fixing. The component C has a linear shape that is long in a predetermined direction as a whole.

Component C has two terminal portions C11 on which multiple electrodes C2 are arranged, and three fixing portions C12 that fix component C. The three fixing portions C12 are arranged in a straight line at intervals. One terminal portion C11 is arranged between the two fixing portions C12 at the center and one end, and another terminal portion C11 is arranged between the two fixing portions C12 at the center and the other end.

The body C1 is made of dark-colored resin. The boss C3 and the screw hole C4 are provided on the bottom surface C13 of the body C1. One boss C3 and one screw hole C4 are provided on each of the three fixing parts C12. The boss C3 is a cylindrical protrusion that protrudes downward from the bottom surface C13 of the body C1. The boss C3 is formed integrally with the body C1 and is made of dark-colored resin. The screw hole C4 is provided in the body C1 by fixing a metal female thread part to a mounting hole formed in the body C1.

The electrodes C2 are lead electrodes that protrude from the body C1 of the component C. The electrodes C2 are made of metal. A large number of electrodes C2 are arranged on each of the two terminal portions C11. Each electrode C2 protrudes diagonally downward from the bottom surface C13 of the body C1 at the terminal portion C11 so as to extend in the width direction of the component C. The tip (bottom end) of the electrode C2 is flat and generally horizontal (parallel to the bottom surface C13 of the body C1) so that it can be connected to the wiring pattern of the substrate P.

In the embodiment, the first recognition target R1 is the tip of a lead electrode (i.e., electrode C2) protruding from the body C1 of the component C. The first recognition target R1 is the tip (lower surface) of each of the multiple electrodes C2. The second recognition target R2 is the tip of a boss C3 protruding from the body C1 made of dark resin. The second recognition target R2 is the tip (lower surface) of each of the three bosses C3.

In the example shown in FIG. 6, the part C is a long part extending in a predetermined direction, and the longitudinal dimension of the part C is larger than the field of view of the imaging device 4. Therefore, the three-dimensional measuring device 1 divides the entire part C into a plurality of regions and performs image recognition (three-dimensional measurement) for each divided region. The number of divided regions of the part C depends on the size of the part C and the field of view size of the imaging device 4. FIG. 7 is an explanatory diagram for explaining imaging of the part C according to the embodiment. FIG. 7 shows an example in which the entire part C is divided into three divided regions. In FIG. 7, the entire part C is divided into divided regions CA1, CA2, and CA3. The divided region CA1 is about 1/3 of the region including one end of the part C. The divided region CA2 is about 1/3 of the region in the center of the part C. The divided region CA3 is about 1/3 of the region including the other end of the part C. The dashed rectangular portion shown in FIG. 7 indicates the field of view range 4S of the imaging device 4. For ease of explanation, FIG. 7 shows the field of view 4S as being located in three divided areas CA1, CA2, and CA3, but in reality, component C is moved by mounting head 22 so that each of divided areas CA1, CA2, and CA3 falls within field of view 4S.

The imaging device 4 separately images the first recognition target R1 and the second recognition target R2 with the divided area CA1 positioned within the field of view 4S. The imaging device 4 separately images the first recognition target R1 and the second recognition target R2 with the divided area CA2 positioned within the field of view 4S. The imaging device 4 separately images the first recognition target R1 and the second recognition target R2 with the divided area CA3 positioned within the field of view 4S. As a result, first image data 61 of the first recognition target R1 in the divided areas CA1, CA2, and CA3 is acquired, and second image data 62 of the second recognition target R2 in the divided areas CA1, CA2, and CA3 is acquired.

In this embodiment, the first image data 61 and the second image data 62 are images of the same area of the part C, but are images captured under different imaging conditions according to the measurement conditions 70 described below.

<Measurement conditions>
The electrode C2 including the first recognition target R1 is made of metal as a whole, and therefore has a relatively high light reflectance. The boss C3 including the second recognition target R2 is made of dark resin integrally formed with the body C1, and therefore has a relatively low light reflectance. In FIG. 7, the dark resin part with low light reflectance (body C1) is colored to explain the difference in light reflectance. However, although the boss C3 (second recognition target R2) is a part with low light reflectance, coloring is omitted from the viewpoint of visibility in the figure. Metal parts such as the electrode C2 have high light reflectance, and therefore saturation (saturation of pixel values) is likely to occur when imaged. A pixel where saturation occurs cannot obtain a correct pixel value (brightness value), and therefore a phase value (height data) cannot be obtained. Therefore, in the three-dimensional measurement of the first recognition target R1, it is important not to generate saturation in the first image data 61. On the other hand, the body C1 including the second recognition target R2 is formed of dark resin so as not to interfere with the recognition of the electrode C2, and is difficult to contrast. Therefore, even if the pattern light PL is projected onto the boss C3, it is difficult to obtain a contrast (brightness distribution) that allows the phase value to be calculated. In this way, the first recognition target R1 and the second recognition target R2 are prone to saturation on the one hand and it is important to suppress excess brightness, while it is difficult to achieve contrast on the other hand and it is important to improve brightness, so it is usually difficult to perform three-dimensional measurement using the same imaging system and the same optical system.

In the embodiment, the measurement conditions 70 (see FIG. 8), which include the imaging conditions and the image recognition conditions, are different for the first recognition target R1 and the second recognition target R2. This makes it possible to perform 3D measurements of both the first recognition target R1 and the second recognition target R2 using the same 3D measurement device 1 (projection device 3 and imaging device 4).

FIG. 8 is a diagram showing a specific example of the measurement condition 70 according to the embodiment. In the example of FIG. 8, the measurement condition 70 includes a camera gain, a threshold, and the number of combined images. The camera gain is the sensitivity of the imaging device 4 (image sensor 42). The higher the camera gain, the higher the luminance value (pixel value) in the image data for the same amount of incident light. The threshold is a threshold applied to threshold processing that divides the recognition target area and the background area in the image. The recognition target area is the area of the first recognition target R1 or the area of the second recognition target R2. The background is the area other than the recognition target. The threshold is set to a value between the range of luminance values of the recognition target and the range of luminance values of the background. The number of combined images is the number of image data to be combined in the combination process described later. The greater the number of combined images, the higher the luminance value (pixel value) of the combined image.

The measurement conditions 70 include a first measurement condition 70A for the first recognition target R1 and a second measurement condition 70B for the second recognition target R2. In the first measurement condition 70A, the camera gain is set to a low value (or a standard value). In the second measurement condition 70B for the second recognition target R2, the camera gain of the imaging device 4 is higher than in the first measurement condition 70A for the first recognition target R1. This increases the luminance value (pixel value) of the second recognition target R2, making it easier to create contrast.

The first measurement condition 70A has a high threshold value (or a standard value). The processing device 5 cuts out an image portion for image recognition from the first image data 61 by threshold processing using the threshold value specified in the first measurement condition 70A. The threshold value of the first measurement condition 70A is set so that an image portion including the entire first recognition target R1 is cut out by the threshold processing. The processing device 5 cuts out an image portion for image recognition from the second image data 62 by threshold processing using the threshold value specified in the second measurement condition 70B. The threshold value of the second measurement condition 70B is set so that an image portion including the entire second recognition target R2 is cut out by the threshold processing. The second measurement condition 70B of the second recognition target R2 has a lower threshold value than the first measurement condition 70A of the first recognition target R1. This reduces the possibility that the area of the second recognition target R2 will be processed as a background area in the second image data 62, in which the luminance value of the second recognition target R2 is difficult to obtain.

In the first measurement condition 70A, the number of combined images is set to a predetermined standard number. In the second measurement condition 70B of the second recognition target R2, the number of combined images is greater than that of the first measurement condition 70A of the first recognition target R1. In one example, the number of combined images in the first measurement condition 70A is set to one, and the number of combined images in the second measurement condition 70B is set to a predetermined number of two or more. Therefore, the processing device 5 performs a combination process of multiple second image data 62 at least in the image recognition of the second recognition target R2. Note that the image combination process is performed when the number of combined images is set to two or more. A number of combined images of one means that no image combination is performed.

9 is a schematic diagram for explaining the image synthesis process according to the embodiment. The projection device 3 projects a stripe pattern with a sine wave-shaped brightness distribution onto the part C while shifting the phase. The projection device 3 projects at least three types of stripe patterns with mutually different phases onto the part C. In one example, the projection device 3 projects four types of stripe patterns with phases differing by π/2 onto the part C. That is, the stripe patterns include a first stripe pattern with a phase shift of 0° (initial phase = 0), a second stripe pattern with a phase shift of 90° (initial phase = π/2), a third stripe pattern with a phase shift of 180° (initial phase = π), and a fourth stripe pattern with a phase shift of 270° (initial phase = 3π/2). The projection device 3 sequentially projects each of the first stripe pattern, the second stripe pattern, the third stripe pattern, and the fourth stripe pattern onto the part C.

Here, image data before image synthesis is called a basic image. Image data after image synthesis is called a synthesized image. When projecting each of the multiple types of stripe patterns, the imaging device 4 captures the basic images the number of image synthesis frames set in the measurement condition 70. If the number of image synthesis frames in the second measurement condition 70B is N, then the basic images are captured N times. Figure 9 shows an example where N=4.

When the first stripe pattern is projected onto the part C, the image data acquisition unit 53 controls the imaging device 4 and the projection device 3 to image the part C onto which the first stripe pattern is projected N times. When the second stripe pattern is projected onto the part C, the image data acquisition unit 53 controls the imaging device 4 to image the part C onto which the second stripe pattern is projected N times. The same applies to the cases where the third stripe pattern is projected onto the part C and the fourth stripe pattern is projected onto the part C. The image data acquisition unit 53 acquires each of the first basic image captured with the first stripe pattern, the second basic image captured with the second stripe pattern, the third basic image captured with the third stripe pattern, and the fourth basic image captured with the fourth stripe pattern in the same number as the number of images to be combined N (=4).

The image data acquisition unit 53 combines the same number of basic images as the number of combined images N to generate a composite image corresponding to each of the multiple types of stripe patterns. The image data acquisition unit 53 combines N first basic images to generate a first composite image corresponding to the first stripe pattern. The image data acquisition unit 53 combines N second basic images to generate a second composite image corresponding to the second stripe pattern. The image data acquisition unit 53 combines N third basic images to generate a third composite image corresponding to the third stripe pattern. The image data acquisition unit 53 combines N fourth basic images to generate a fourth composite image corresponding to the fourth stripe pattern.

In an embodiment, combining the multiple basic images includes adding the multiple basic images. The adding process of the multiple basic images includes adding the luminance values of the same pixel in the multiple basic images. This results in pixels with high luminance values in the combined image. In each combined image, the luminance difference (i.e., contrast) between the light and dark areas of the stripe pattern becomes large. This makes it easier to calculate the phase value even in the part of the second recognition target R2 where contrast is difficult to achieve.

The above measurement conditions 70 are stored in the memory unit 56. When performing three-dimensional measurement of the first recognition target R1, the processing device 5 acquires the first measurement conditions 70A from the memory unit 56 and performs imaging control and image recognition based on the first measurement conditions 70A. When performing three-dimensional measurement of the second recognition target R2, the processing device 5 acquires the second measurement conditions 70B from the memory unit 56 and performs imaging control and image recognition based on the second measurement conditions 70B.

<Image Recognition>
10 is an explanatory diagram for explaining a recognition algorithm of the image recognition process executed by the processing device 5 of the embodiment. The processing device 5 executes different recognition algorithms for the first recognition target R1 and the second recognition target R2 to perform image recognition.

When performing three-dimensional measurement of the first recognition target R1, the phase value calculation unit 54 executes the electrode recognition algorithm 71A. The electrode recognition algorithm 71A is optimized for recognizing the first recognition target R1, i.e., the tip of the electrode C2 of the component C. The electrode recognition algorithm 71A is suitable for recognizing the first recognition target R1 of a known shape based on the component data. The shape of the first recognition target R1 is, for example, rectangular.

When performing three-dimensional measurement of the second recognition target R2, the phase value calculation unit 54 executes the boss recognition algorithm 71B. The boss recognition algorithm 71B is optimized for recognizing the second recognition target R2, i.e., the tip of the boss C3 formed on the body C1 of the part C. The boss recognition algorithm 71B is suitable for recognizing the second recognition target R2 of a known shape based on the part data. The shape of the second recognition target R2 is, for example, circular.

The electrode recognition algorithm 71A and boss recognition algorithm 71B are stored in the memory unit 56. The processing device 5 selectively reads out the recognition algorithm corresponding to the recognition target from the memory unit 56 and executes image recognition. The phase value calculation unit 54 generates measurement data representing the phase value of each pixel constituting the recognition target (first recognition target R1 and second recognition target R2) through image recognition.

<Complementary processing>
FIG. 11 is an explanatory diagram showing an example of the first complementation process of image recognition by the processing device 5 of the embodiment. As described above, in the embodiment, the projection device 3 can irradiate the pattern light PL from four directions, and four image data with different irradiation directions of the pattern light PL are acquired. That is, acquisition of the basic image shown in FIG. 9, generation of a composite image (only when performing a composite process), and calculation of a three-dimensional shape are performed four times for each direction. As a result, the processing device 5 generates four types of measurement data for each direction, as shown in FIG. 11. FIG. 11 shows an example of first direction measurement data 81A, second direction measurement data 81B, third direction measurement data 81C, and fourth direction measurement data 81D generated by image recognition of one recognition target.

In each measurement data, an error region 82 may occur where height data (or phase value) cannot be calculated due to differences in the irradiation direction of the pattern light PL. The error region 82 is a saturation region that occurs when specularly reflected light is directly incident on the imaging device 4, or a shadow region where light is blocked by other parts of the part C, depending on the relationship between the shape and orientation of the recognition target and the irradiation direction of the pattern light PL.

In the embodiment, when the processing device 5 fails to recognize any part of the image data obtained by the pattern light PL in multiple directions, the processing device 5 performs image recognition based on the other image data that has been successfully recognized. In other words, even if an error region 82 occurs in any of the first direction measurement data 81A, the second direction measurement data 81B, the third direction measurement data 81C, and the fourth direction measurement data 81D, if the processing device 5 is able to calculate height data for the same region in the measurement data in the other directions, the processing device 5 performs a first complementation process that complements the measurement result (height data) based on the height data.

For example, assume that the circular shape shown in the top row of Figure 11 is the true shape of the recognition target (e.g., second recognition target R2). In each piece of measurement data shown in the middle row of Figure 11, the white area indicates the recognition area 83 where height data was calculated (recognition was successful). In each piece of measurement data, the area surrounded by a dotted line adjacent to the recognition area 83 (the area of difference between the true shape and the recognition area 83) is the error area 82. In each piece of measurement data, the area outside the recognition area 83 and error area 82 (black area) is the background part.

In the example of FIG. 11, an error region 82 is formed on the bottom side of the figure in the first direction measurement data 81A, an error region 82 is formed on the top side of the figure in the second direction measurement data 81B, an error region 82 is formed on the right side of the figure in the third direction measurement data 81C, and an error region 82 is formed on the left side of the figure in the fourth direction measurement data 81D. In the example of FIG. 11, an error region 82 is partially formed in each of the measurement data, and there is no measurement data that was able to obtain the true shape of the recognition target.

The processing device 5 synthesizes the measurement data in each direction by a method of calculating the logical sum of each recognition area 83 of the first direction measurement data 81A, the second direction measurement data 81B, the third direction measurement data 81C, and the fourth direction measurement data 81D, and generates the synthesized measurement data 80 shown in the lower part of FIG. 11. For example, the processing device 5 generates height data of a position (pixel) belonging to the error area 82 of the first direction measurement data 81A using height data of the recognition area 83 of the second direction measurement data 81B, the third direction measurement data 81C, and the fourth direction measurement data 81D. When multiple height data are obtained for the same position, the processing device 5 obtains a representative value (for example, an average value) of the multiple height data and sets it as the height data of that position. When only one height data is obtained for the same position, the processing device 5 adopts the value of that height data as the height data of that position. As a result, in the synthesized measurement data 80, height data is obtained for the entire area corresponding to the true shape of the recognition target shown in the upper part of FIG. 11.

Figure 12 is an explanatory diagram showing an example of the second complementation process of image recognition by the processing device 5 of the embodiment. The above-mentioned first complementation process is applied to the error region 82 where image recognition was successful in at least one of the measurement data in the four directions and height data (phase value) was obtained. On the other hand, for example, when the tip of the electrode C2 (first recognition target R1) is mirror-like, saturation may occur, making it impossible to obtain height data (phase value) in all of the measurement data in the four directions.

In the embodiment, the processing device 5 executes a second complementation process when it fails to recognize the saturation region 84 because the saturation region 84 is included in part of the first image data 61. In the second complementation process, the processing device 5 determines whether or not the saturation region 84 is part of the first recognition target R1 based on the shape data of the part C. Then, when the processing device 5 determines that the saturation region 84 is part of the first recognition target R1, it executes a process of complementing the saturation region 84 using the recognition result of the first recognition target R1. The saturation region 84 is a pixel region in the image data (first image data 61) where the brightness value (pixel value) reaches the maximum value.

Specifically, as shown in FIG. 12, assume that a saturation region 84 exists in a part of the first image data 61. In the first image data 61 in FIG. 12, the phase of the stripe pattern differs between the central rectangular area and the outer periphery. The central rectangular area indicates the first recognition target R1 (i.e., the lower surface of the electrode C2), and the outer periphery is the background area BG. In FIG. 12, a part of the left end of the first recognition target R1 in the figure is the saturation region 84. Since the pixel value (brightness value) of each pixel included in the saturation region 84 is the maximum value, the phase of the stripe pattern cannot be recognized. FIG. 12 illustrates the first image data 61 in one irradiation direction, but the saturation region 84 is formed at the same position in the first image data 61 in the other irradiation direction.

In this case, in the measurement data for each direction (first direction measurement data 81A to fourth direction measurement data 81D), height data (phase value) cannot be obtained in the saturation region 84, so it becomes an error region 82. In FIG. 12, only the first direction measurement data 81A is illustrated. Of the first recognition target R1, the area other than the error region 82 becomes a recognition region 83 from which height data (phase value) was obtained. If the saturation region 84 becomes an error region 82 in all of the measurement data for the four directions (first direction measurement data 81A to fourth direction measurement data 81D), the first complementation process described above cannot be applied.

In this case, the processing device 5 performs a process of determining whether the saturation region 84 is part of the first recognition target R1 based on the part data 72 stored in the storage unit 56. The processing device 5 identifies the part of the part C that is reflected in the first image data 61 based on the relationship between the position of the first recognition target R1 obtained from the part data 72, the position of the background region BG (such as the body C1), and the position of the imaging device 4. In other words, the processing device 5 performs a calculation to determine whether the first recognition target R1 is present on the optical path extended along the optical axis AX (see FIG. 2) from the position corresponding to the pixel position of the saturation region 84, and if the first recognition target R1 is present, it determines that the saturation region 84 is part of the first recognition target R1. If the first recognition target R1 is not present on the optical path, the processing device 5 determines that the saturation region 84 is not part of the first recognition target R1. In this embodiment, the imaging optical system 41 is a telecentric optical system, which, unlike other optical systems in which the chief ray is not parallel to the optical axis AX, does not require correction of the lens angle of view, etc., and therefore reduces the amount of calculation required for the determination process.

When the processing device 5 determines that the saturation region 84 is part of the first recognition target R1, it complements the height data (phase value) of the saturation region 84 with the height data (phase value) of the other recognition region 83 of the first recognition target R1. As a result, in the measurement data 80 after the second complementation process, the overall height data of the first recognition target R1, including the saturation region 84, is calculated.

On the other hand, if the processing device 5 determines that the saturation region 84 is not part of the first recognition target R1, the saturation region 84 remains as an error region 82. In this case, in the measurement data 80 after the second complementation process, the portion of the saturation region 84 is processed as a background region BG.

<3D measurement method>
The operation of the three-dimensional measuring apparatus 1 according to the embodiment will be described below. Fig. 13 is a flowchart showing a three-dimensional measuring method according to the embodiment. The three-dimensional measuring method according to the embodiment, which is executed by the three-dimensional measuring apparatus 1, will be described with reference to Fig. 13.

As shown in FIG. 13, the three-dimensional measurement method according to the embodiment generally includes an imaging step S10, an image synthesis step S20, an image recognition step S30, and an interpolation processing step S40.

In the imaging step S10, the three-dimensional measuring device 1 performs a first imaging (step S11) of the first recognition target R1 of the part C, and a second imaging (step S12) of the second recognition target R2. In FIG. 13, the processing is performed in the order of the first imaging (step S11) and the second imaging (step S12), but these processing may be performed in the reverse order, i.e., the second imaging and the first imaging.

First, the 3D measuring device 1 performs a first image capture (step S11) with one of the divided areas of the component C (e.g., divided area CA1) positioned within the field of view 4S by the mounting head 22. The processing device 5 performs control to acquire the first image data 61 of the first recognition target R1 based on the first measurement conditions 70A.

The processing device 5 controls the first and second projection devices 3 to emit the first pattern light PL1 and the second pattern light PL2 based on the first pattern data and the second pattern data created by the pattern generation unit 52, respectively, and irradiate them toward the part C. The image data acquisition unit 53 acquires the first image data 61 of the part C onto which the first pattern light PL1 and the second pattern light PL2 are projected in four directions, respectively, with a camera gain specified in the first measurement condition 70A. The image data acquisition unit 53 acquires each of the first image data 61 in the four directions, the number of images equal to the number of image combinations specified in the first measurement condition 70A. The number of image combinations N in the first measurement condition 70A is, for example, 1 (no image combination).

The processing device 5 controls the first and second projection devices 3 to obtain first image data 61 in which four types of stripe patterns with different phases, from the first stripe pattern to the fourth stripe pattern, are projected. In other words, for each of the four irradiation directions, the first image data 61 of the four phase patterns is obtained for each of the image synthesis number N (=1).

The three-dimensional measuring device 1 performs a second image capture (step S12) for the same divided area (e.g., divided area CA1). The processing device 5 performs control to acquire second image data 62 of the second recognition target R2 based on the second measurement conditions 70B.

The processing device 5 controls the first and second projection devices 3 to emit the first pattern light PL1 and the second pattern light PL2, respectively, and irradiate them toward the part C. The image data acquisition unit 53 acquires the second image data 62 of the part C onto which the first pattern light PL1 and the second pattern light PL2 are respectively projected in four directions, with a camera gain defined in the second measurement condition 70B. The image data acquisition unit 53 acquires each of the second image data 62 in the four directions, the number of images equal to the number of combined images N defined in the second measurement condition 70B. The number of combined images N in the second measurement condition 70B is, for example, four. This allows the basic image of the second image data 62 to be captured.

The processing device 5 controls the first and second projection devices 3 to obtain second image data 62 in which four types of stripe patterns with different phases, from the first stripe pattern to the fourth stripe pattern, are projected. In other words, for each of the four irradiation directions, first image data 61 (basic images) of the four phase patterns are obtained for each of the image synthesis number N (=4).

The processing device 5 determines whether all image data to be acquired has been acquired (step S13). The processing device 5 determines that all image data has been acquired when the imaging of the three divided areas CA1, CA2, and CA3 of the component C is complete. The processing device 5 determines that all image data has not been acquired if there are divided areas that have not been imaged. In this case, the processing device 5 controls the mounting head 22 by the control device 29 to change the position of the component C so that the next divided area (e.g., divided area CA2) is positioned within the field of view range 4S (step S14).

The processing device 5 performs a first image capture (step S11) and a second image capture (step S12) for the divided area CA2. After step S13, the processing device 5 changes the image capture position so that the next divided area (e.g., divided area CA3) is positioned within the field of view 4S (step S14). Similarly, the processing device 5 performs a first image capture (step S11) and a second image capture (step S12) for the divided area CA3. As a result, imaging is completed for all divided areas, and the processing device 5 proceeds to the image synthesis step S20 after step S13.

When the number of combined images under the measurement conditions is two or more, the processing device 5 performs a combination process on the image data (image combination step S20). In the embodiment, the number of combined images N is set to 1 under the first measurement conditions 70A, so the processing device 5 does not perform image combination on the first image data 61. The processing device 5 generates four sets of images of the first to fourth stripe patterns as the first image data 61, four sets for each irradiation direction.

Since the number of image combinations N=4 is set in the second measurement condition 70B, the processing device 5 performs image combination processing on the second image data 62. As shown in FIG. 9, the processing device 5 combines N basic images of each of the first to fourth stripe patterns to generate a fourth composite image from the first composite image. The processing device 5 performs combination processing of each basic image of the first to fourth stripe patterns for each of the second image data 62 in the four irradiation directions. The processing device 5 generates four sets of the first to fourth composite images for each irradiation direction as the second image data 62.

The processing device 5 performs image recognition on the first image data 61 and the second image data 62 (image recognition step S30). The phase value calculation unit 54 calculates the phase value of each of the multiple pixels in the image data based on the luminance (pixel value) of each image data of the first stripe pattern to the fourth stripe pattern for each irradiation direction. The phase value calculation unit 54 cuts out the recognition target area for the first image data 61 using a threshold value specified in the first measurement condition 70A, and calculates the height data (phase value) of the first recognition target R1 by executing the electrode recognition algorithm 71A. The processing device 5 cuts out the recognition target area for the second image data 62 using a threshold value specified in the second measurement condition 70B, and calculates the phase value of the second recognition target R2 by executing the boss recognition algorithm 71B.

The three-dimensional shape calculation unit 55 calculates height data for each pixel of the image data corresponding to each recognition target based on the phase value and the principle of triangulation. Here, there is a one-to-one correspondence between the height data for each pixel of the image data and the height data for each point in the recognition target of part C. The height data for each point in the recognition target of part C indicates the coordinate values of each point in the recognition target of part C in three-dimensional space. The three-dimensional shape calculation unit 55 generates measurement data for the first recognition target R1 and measurement data for the second recognition target R2.

When an error region 82 exists in the measurement data of the first recognition target R1 and the measurement data of the second recognition target R2, the processing device 5 executes a complementation process for pixels belonging to the error region 82 (complementation process step S40). When an error region 82 exists in any part of the measurement data for each of the four irradiation directions, and there is measurement data for other irradiation directions that successfully recognize the same region, the processing device 5 executes a first complementation process. When the processing device 5 fails to recognize the saturation region 84 in all measurement data for the four irradiation directions due to the saturation region 84 being included in part of the first image data 61, the processing device 5 executes a second complementation process in which the processing device 5 determines whether the saturation region 84 is part of the first recognition target R1, and when it is determined that the saturation region 84 is part of the first recognition target R1, it executes a second complementation process in which the processing device 5 executes a process of complementing the saturation region 84 using the recognition result of the first recognition target R1. Note that, when the error region 82 (saturation region 84) does not exist, the processing device 5 does not execute the first and second complementation processes.

As a result, the processing device 5 calculates the three-dimensional shapes of the first recognition target R1 and the second recognition target R2 based on the height data at each point of the first recognition target R1 and the second recognition target R2.

<Implementation method>
14 is a flowchart illustrating the operation of the electronic component mounting system 100 according to the embodiment. The operation of the electronic component mounting system 100 according to the embodiment will be described below. As shown in FIG. 14, the electronic component mounting system 100 executes a detection step S60, a proximity step S70, and an insertion step S80.

The three-dimensional measuring device 1 recognizes the first recognition target R1 (electrode C2) and second recognition target R2 (boss C3) of the component C by executing the three-dimensional measuring method according to the embodiment. The processing device 5 transmits measurement data of the first recognition target R1 and the second recognition target R2 to the control device 29. The control device 29 adjusts the relative position between the component C and the substrate P based on the height data of the first recognition target R1 and the second recognition target R2, and controls the mounting head 22 to mount the component C on the substrate P (detection step S60).

Based on a command from the 3D measuring device 1, the control device 29 controls the head driving device 25 and nozzle driving device 26 of the mounting head 22 to bring the lower vertical tip of the component C held by the nozzle 11 close to the top surface of the substrate P (close step S70).

In the proximity step S70, the control device 29 controls the nozzle driving device 26 of the mounting head 22 to move the nozzle 11 vertically downward so that the vertically lower tip of the component C held by the nozzle 11 approaches the top surface of the substrate P based on the height data of the first recognition target R1 and the second recognition target R2 of the component C received from the 3D measurement device 1.

In the approach step S70, the control device 29 also controls the head driving device 25 and nozzle driving device 26 of the mounting head 22 to move the mounting head 22 horizontally and rotate the nozzle 11 vertically so that the positions of the first recognition target R1 and the second recognition target R2 of the component C held by the nozzle 11 coincide with the electrode connection positions and the horizontal positions of the through holes formed on the substrate P, based on the measurement data of the three-dimensional shape of the component C held by the nozzle 11 and the measurement data of the position measuring device 27 indicating the positions of the through holes formed on the substrate P.

Based on a command from the three-dimensional measuring device 1, the control device 29 controls the nozzle driving device 26 of the mounting head 22 to insert the component C held by the nozzle 11 into a through hole formed in the substrate P (insertion step S80). The control device 29 controls the nozzle driving device 26 of the mounting head 22 to move the nozzle 11 downward in the vertical direction. As a result, the three bosses C3 of the component C are inserted into the corresponding through holes of the substrate P, and the component C is positioned on the substrate P. As a result, each electrode C2 of the component C is positioned at the electrode connection position of the substrate P. Note that each electrode C2 of the component C may be a pin-type electrode that is inserted into an electrode insertion hole of the substrate P.

＜Effects＞
As described above, according to this embodiment, the three-dimensional measuring apparatus 1 includes the projection device 3 that projects pattern light PL with different phases onto a part C having a first recognition target R1 and a second recognition target R2 having a lower light reflectance than the first recognition target R1, the imaging device 4 that separately images the first recognition target R1 and the second recognition target R2 onto which the pattern light PL is projected, and the processing device 5 that performs image recognition of the first recognition target R1 based on first image data 61 of the first recognition target R1 and image recognition of the second recognition target R2 based on second image data 62 of the second recognition target R2, and the measurement conditions 70 including the imaging conditions and the image recognition conditions are different for the first recognition target R1 and the second recognition target R2. As a result, the first recognition target R1, which has a relatively high light reflectance, and the second recognition target R2, which has a low light reflectance, are imaged separately under measurement conditions suitable for each, so that the first image data 61 suitable for image recognition of the first recognition target R1, which is prone to saturation, and the second image data 62 suitable for image recognition of the second recognition target R2, which is difficult to create contrast, are obtained. Then, the first image data 61 and the second image data 62 are image-recognized under measurement conditions suitable for each, so that the first recognition target R1 and the second recognition target R2 can be three-dimensionally measured. As a result, even if one part includes a recognition target with a high light reflectance and a recognition target with a low light reflectance, each recognition target can be appropriately recognized by the same measurement device.

According to this embodiment, the second measurement condition 70B, which is the measurement condition of the second recognition target R2, has a higher camera gain of the imaging device 4 than the first measurement condition 70A, which is the measurement condition of the first recognition target R1. The second measurement condition 70B has a lower threshold value for dividing the recognition target area and the background area in the image than the first measurement condition 70A. The second measurement condition 70B has a larger number of image composites than the first measurement condition 70A. As a result, the first image data 61 of the first recognition target R1, which is prone to saturation, can be generated with a relatively low gain, a high threshold value, and a small number of composites. As a result, saturation in the image data of the first recognition target R1 can be suppressed, and the first recognition target R1 and the background can be distinguished more accurately. The second image data 62 of the second recognition target R2, which is difficult to create contrast, can be generated with a relatively high gain, a low threshold value, and a large number of composites. As a result, the image data of the second recognition target R2 is more likely to have contrast, and the second recognition target R2 can be distinguished more accurately from the background.

According to this embodiment, if the processing device 5 fails to recognize any part of the image data generated by the pattern light PL in multiple directions, the processing device 5 performs image recognition (first complementation process) based on the other image data that was successfully recognized. As a result, even if image recognition fails for part of the recognition target in the image data and an error area 82 occurs, if image recognition is successful with image data from other irradiation directions, the measurement data can be complemented to suppress the occurrence of the error area 82, which is a partial lack of measurement data.

According to this embodiment, when the processing device 5 fails to recognize the saturation region 84 due to the saturation region 84 being included in part of the first image data 61, the processing device 5 determines whether the saturation region 84 is part of the first recognition target R1 based on the shape data of the part C, and when it determines that the saturation region 84 is part of the first recognition target R1, it performs a process (second complementation process) to complement the saturation region 84 using the recognition result of the first recognition target R1. As a result, even if the saturation region 84 occurs in a part of the first recognition target R1 with high light reflectance in the first image data 61 and measurement data cannot be obtained, the recognition result of the saturation region 84 can be complemented. As a result, even if the first recognition target R1 is prone to saturation due to its high light reflectance, the contour shape of the first recognition target R1 can be accurately obtained.

According to this embodiment, the first recognition target R1 is the tip of the electrode C2 protruding from the body C1 of the component C. The second recognition target R2 is the tip of the boss C3 protruding from the body C1 made of dark-colored resin. This allows image recognition to be performed together with the electrode C2 using the same measuring device, even on a part of the body C1 made of dark-colored resin that does not have much contrast, so as not to interfere with image recognition of the electrode C2.

[Other embodiments]
In the above-described embodiment, an example was shown in which the three-dimensional measuring device 1 was provided in an electronic component mounting system 100 equipped with an electronic component mounting device 2, but the three-dimensional measuring device 1 may also be provided in, for example, an inspection device that inspects components C.

1...3D measurement device, 2...electronic component mounting device, 3...projection device, 4...imaging device, 4S...field of view, 5...processing device, 6...reflective member, 7...housing, 11...nozzle, 22...mounting head, 23...electronic component supply device, 24...substrate transport device, 25...head drive device, 26...nozzle drive device, 27...position measurement device, 29...control device, 31...light source, 32...light modulation element, 33...emission optical system, 33S...emission surface, 41...imaging optical system, 41S...incident surface, 42...imaging element, 51...input/output unit, 52...pattern generation unit, 53...image data acquisition unit, 54...phase value calculation unit, 55...3D shape calculation unit, 56...storage unit, 61...first image data, 62...second image data, 70...measurement conditions, 70A...first measurement conditions, 70B...second measurement conditions, 71A...electrode recognition algorithm, 71B... boss recognition algorithm, 72... component data, 80... measurement data, 81A... first direction measurement data, 81B... second direction measurement data, 81C... third direction measurement data, 81D... fourth direction measurement data, 82... error region, 83... recognition region, 84... saturation region, 100... electronic component mounting system, 331... first region, 332... second region, AX... optical axis, BG... background Area, C... component, C1... body, C2... electrode, C3... boss, C4... screw hole, C11... terminal part, C12... fixing part, C13... bottom surface, CA1, CA2, CA3... divided area, MP... mounting position, N... number of images to be combined, P... board, PL... pattern light, PL1... first pattern light, PL2... second pattern light, R1... first recognition target, R2... second recognition target, SP... supply position, TP... measurement position.

Claims (8)

A projection device that projects pattern light having different phases onto a part having a first recognition target and a second recognition target having a lower light reflectance than the first recognition target;
an imaging device that separately images the first recognition target and the second recognition target onto which the pattern light is projected;
a processing device that performs image recognition of the first recognition target based on first image data of the first recognition target, and performs image recognition of the second recognition target based on second image data of the second recognition target,
Measurement conditions including imaging conditions and image recognition conditions are different between the first recognition target and the second recognition target.
3D measuring device. the measurement condition of the second recognition target is higher than the measurement condition of the first recognition target, and a camera gain of the imaging device is higher.
The three-dimensional measuring apparatus according to claim 1 . The measurement condition of the second recognition target has a lower threshold value for dividing a recognition target region and a background region in an image than the measurement condition of the first recognition target.
The three-dimensional measuring apparatus according to claim 1 . The measurement condition of the second recognition target includes a larger number of times of imaging than the measurement condition of the first recognition target,
The processing device performs a synthesis process on the plurality of second image data.
The three-dimensional measuring apparatus according to claim 1 . the projection device projects the pattern light onto the component from a plurality of directions separately,
the imaging device captures images of the pattern light from the plurality of directions separately,
when the processing device fails to recognize any part of the image data obtained by the pattern light in the multiple directions, the processing device performs image recognition based on other image data that has been successfully recognized.
The three-dimensional measuring apparatus according to claim 1 . When the processing device fails to recognize the saturation region due to a saturation region being included in a part of the first image data,
determining whether the saturation region is a part of the first recognition target based on shape data of the part;
When it is determined that the saturation region is a part of the first recognition target, a process of complementing the saturation region using a recognition result of the first recognition target is performed.
The three-dimensional measuring apparatus according to claim 1 . the first recognition target is a tip portion of an electrode protruding from a body of the component,
the second recognition target is a tip portion of a boss protruding from the body made of a dark-colored resin;
The three-dimensional measuring apparatus according to claim 1 . A step of projecting pattern light having different phases onto a component having a first recognition target and a second recognition target having a lower light reflectance than the first recognition target, and capturing a first image of the first recognition target onto which the pattern light is projected;
projecting the pattern light onto the component and capturing a second image of the second recognition target onto which the pattern light is projected;
performing image recognition on the first recognition target based on first image data of the first recognition target, and performing image recognition on the second recognition target based on second image data of the second recognition target;
Measurement conditions including imaging conditions and image recognition conditions are different between the first recognition target and the second recognition target.
3D measurement method.

Images