JP2017182113A - Work determination apparatus and method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a work determination device can determine whether a work is arranged at a loading site in a state where a front/rear of the work is correctly put in place.SOLUTION: A template image edge detection unit 34 is configured to generate a template edge image for making the template edge image perform pattern matching relative to a work arranged in a loading site, and a work shot image edge detection unit 35 is configured to generate a work shot edge image of work shot image data having a work shot. A work location detection unit 361 is configured to make template edge image match the work shot edge image to detect a position of the work. A front/rear determination-purpose characteristic image extraction unit 38 is configured to generate a template rear image; subtract an edge where a template image and a template rear image overlap with each other from the template image to extract a front/rear determination-purpose characteristic image. A front/rear determination unit 362 is configured to move the front/rear determination-purpose characteristic image to a position of the work; calculate a similarity degree between the work shot edge image and the front/rear determination-purpose characteristic image, and determine a front and rear of the work.SELECTED DRAWING: Figure 4

Description

  The present invention relates to a workpiece determination apparatus and method that can determine whether a workpiece is placed in a correct state at a placement place.

  When a metal workpiece is machined by a processing machine such as a bending machine, a workpiece placed at a predetermined placement location may be held by a workpiece holding robot and carried into the machining machine. In this case, the image calculation processing device performs pattern matching between the template image for pattern matching and a photographed image obtained by photographing the work placed at the placement place with the camera, so that where the work is located in the placement place. It can be detected whether it is arranged in such an orientation.

JP 7-319525 A JP 2000-288974 A

  An operator may mistakenly place the work face down on the placement location. If the feature difference between the front and back surfaces of the workpiece is small, the matching rate is determined by pattern matching between the template image and the captured image of the workpiece between the workpiece placed face up and the workpiece placed face down. There may be no big difference.

  Then, the work arranged face down is held by the work holding robot and processed by the processing machine. A product obtained by processing a work placed face down will be a defective product.

  In order to solve such a problem, it is conceivable to prepare template images for the front surface and the back surface of the workpiece, and pattern-match the photographed image of the workpiece with the template images for the front surface and the back surface. If the matching rate by pattern matching between the photographed image of the workpiece and the template image for the front surface is higher than the matching rate by pattern matching between the photographed image of the workpiece and the template image for the back surface, the workpiece is correctly placed face up. Can be determined.

  However, the method of performing pattern matching between the photographed image of the workpiece and the template images for the front and back surfaces requires two times the processing time required for pattern matching, which is inefficient. There is a need for a method for efficiently and accurately determining the front and back of a workpiece without significantly increasing the processing time required for pattern matching.

  An object of this invention is to provide the workpiece determination apparatus and method which can determine efficiently and correctly whether the workpiece | work is arrange | positioned in the state where the front and back are correct in the mounting place.

  In order to solve the above-described problems of the related art, the present invention detects a template image edge for pattern matching with a workpiece placed at a placement location, and generates a template edge image. A workpiece photographed image edge detection unit that detects an edge of workpiece photographed image data obtained by photographing the workpiece placed at the above-described placement location, and generates a workpiece photographed edge image, the template edge image, and the workpiece photographed A workpiece position detection unit that detects the position of the workpiece arranged at the placement location by matching with an edge image, and generates a template back image showing the back side of the workpiece by inverting the front and back of the template image From the template image, the template image and the template back image are A feature image extracting unit for front / back determination for extracting a front / back determination feature image for determining the front / back of the workpiece by subtracting improper edges, and the feature image for front / back determination detected by the workpiece position detection unit. The workpiece is moved to the position of the workpiece, the degree of similarity between the workpiece photographing edge image and the feature image for front / back determination is calculated, and the front / back of the workpiece arranged at the above-described placement location is determined based on the similarity. Provided is a workpiece determination device including a front / back determination unit.

  In addition, in order to solve the above-described problems of the conventional technology, the present invention detects a template image edge for pattern matching with a workpiece placed at a placement place, and generates a template edge image. An edge detection step, a workpiece photographed image edge detection step for generating a workpiece photographed edge image by detecting an edge of the workpiece photographed image data obtained by photographing the workpiece arranged at the above-described placement location, the template edge image, and the template image A workpiece position detecting step for detecting the position of the workpiece arranged at the placement location by matching the workpiece photographing edge image, and a template back image showing the back side of the workpiece by inverting the front and back of the template image And generating the template image and the template from the template image. A feature image extracting step for front / back determination for extracting a front / back determination feature image for determining the front / back of the workpiece by subtracting an edge that overlaps the back / back image, and the feature image for front / back determination to the workpiece position detection step. To the position of the workpiece detected in this way, calculate the similarity between the workpiece photographing edge image and the front / back feature image, and based on the similarity, There is provided a workpiece determination method including a front / back determination step of determining a front / back side.

  According to the workpiece determination apparatus and method of the present invention, it is possible to efficiently and accurately determine whether a workpiece is placed in a correct state at the place where the workpiece is placed.

It is a perspective view showing the example of the whole composition of a processing system. It is a perspective view which shows an example of the workpiece | work processed with a processing system. 1 is a block diagram illustrating an example of an overall configuration of a machining system including a workpiece determination device and a workpiece holding robot control device according to an embodiment. FIG. 4 is a block diagram illustrating an internal configuration example of the image arithmetic processing device 3 and the robot control device 5 of FIG. 3. It is a 1st partial flowchart for demonstrating the operation | movement of the workpiece | work determination apparatus of one Embodiment, and the workpiece | work determination method of one Embodiment. It is a 2nd partial flowchart for demonstrating the operation | movement of the workpiece | work determination apparatus of one Embodiment, and the workpiece | work determination method of one Embodiment. It is a 3rd partial flowchart for demonstrating the operation | movement of the workpiece | work determination apparatus of one Embodiment, and the workpiece | work determination method of one Embodiment. It is a 4th partial flowchart for demonstrating the operation | movement of the workpiece | work determination apparatus of one Embodiment, and the workpiece | work determination method of one Embodiment. It is a figure which shows an example of the workpiece | work imaging | photography image used with the workpiece | work determination apparatus and method of one Embodiment. It is a figure which shows the template image used with the workpiece | work determination apparatus and method of one Embodiment. It is a figure which shows the filling template image used with the workpiece | work determination apparatus and method of one Embodiment. It is a figure which shows the template back image used with the workpiece | work determination apparatus and method of one Embodiment. It is a figure which shows the state which matched the fill template image and the template back image. It is a figure which shows the overlapping edge of a filling template image and a template back image. It is a figure which shows the feature image for front / back determination extracted by subtracting an overlapping edge from a filling template image. It is a figure which shows an example of the coefficient matrix used in order to detect the edge of a filling template image and a workpiece | work imaging | photography image. It is a figure for demonstrating the edge strength of the edge of a filling template image or a workpiece | work imaging | photography image. It is a figure for comparing the state which carried out Sobel conversion of each of a template image and a filling template image. FIG. 5 is a conceptual diagram illustrating a state in which the template edge image and the workpiece photographing edge image are matched, and the position of the template edge image is at a position where the similarity between the template edge image and the workpiece photographing edge image is relatively small. . FIG. 6 is a conceptual diagram illustrating a matching process between a template edge image and a workpiece photographing edge image, and showing a state where the position of the template edge image is at a position where the similarity between the template edge image and the workpiece photographing edge image is relatively large. . It is a figure which shows the workpiece | work imaging | photography edge image when the workpiece | work is arrange | positioned face-up on a loading stand, and the workpiece | work imaging | photography edge image when a workpiece | work is arrange | positioned face-down on a loading stand. It is a figure which shows expansion and rotation of the feature image for front / back determination. A process of calculating the similarity between the workpiece photographing edge image when the workpiece is placed face up on the loading table and the enlarged and rotated feature image for front / back determination, and the workpiece is placed face down on the loading table. It is a figure for demonstrating the process which calculates the similarity degree of the workpiece | work imaging | photography edge image and the enlarged and rotated feature image for front and back determination.

  Hereinafter, a workpiece determination apparatus and method according to an embodiment will be described with reference to the accompanying drawings.

  First, an example of the overall configuration of a machining system configured to hold a workpiece placed at a placement place with a workpiece holding robot and carry it into a processing machine will be described with reference to FIG. In the configuration example shown in FIG. 1, the processing machine 10 is a bending machine, and the workpiece holding robot 20 is a workpiece transfer robot that sucks or grips the workpiece and transfers it to the bending machine.

  The processing machine is not limited to a bending machine, and may be any processing machine such as a punching machine or a laser machine.

  The processing machine 10 includes an upper table 101 that is movable in the vertical direction and a fixed lower table 102. A punch die Tp is attached to the upper table 101, and a die die Td is attached to the lower table 102. The processing machine 10 is equipped with a display / operation unit 21 (operation pendant) connected to an NC device 2 (shown in FIG. 3) described later.

  An articulated workpiece holding robot 20 is installed in front of the processing machine 10. The workpiece holding robot 20 is disposed on the guide rail 201 and is configured to be movable along the guide rail 201.

  In front of the workpiece holding robot 20, a loading table 30, which is a place where the workpiece W is placed, is installed. The workpiece W is a metal plate material having a predetermined shape as shown in FIG. Here, the workpiece W has a rectangular shape and has a partial cut-and-raised portion W0. The cut-and-raised portion W0 has a rectangular hole W01 that is elongated and cut, and cuts W02 and W03 that are connected to both ends of the hole W01 and extend in a direction perpendicular to the longitudinal direction of the hole W01. A plurality of workpieces W are stacked on the loading table 30.

  The plurality of workpieces W are not necessarily accurately stacked on the loading table 30 so that the positions in the surface direction do not shift, but are stacked with the positions in the surface direction shifted as shown in FIG. May have. It is assumed that the state in which the workpiece W is arranged as shown in FIG. 1 is the state in which the workpiece W is correctly arranged face up.

  A column 401 extending to a predetermined height with respect to the installation surface of the loading table 30 is disposed in the vicinity of the loading table 30. The support column 401 extends in a vertical direction up to a predetermined height, is bent in the horizontal direction toward the loading table 30, and is further bent in the direction of the processing machine 10. The camera 4 is attached to the tip of the column 401.

  The camera 4 is configured to photograph the workpiece W placed on the loading table 30. Since the position of the workpiece W on the loading table 30 may be off, the camera 4 should take an image of the entire loading table 30. For example, the position of the corner P0 of the loading table 30 is set as the origin of the captured image obtained by the camera 4 capturing the loading table 30. The origin position of the workpiece holding robot 20 is set to coincide with the origin of the photographed image.

  The configuration and operation of a control device that controls the processing machine 10 and the workpiece holding robot 20 will be described with reference to FIG. In FIG. 3, CAM 1 holds CAD data indicating a work W generated by a CAD (not shown). The NC device 2 controls the processing machine 10. The display / operation unit 21 is connected to the NC device 2.

  An image arithmetic processing device 3 and a robot control device 5 are connected to the NC device 2. The image arithmetic processing device 3 constitutes a work determination device of the present embodiment. The image arithmetic processing device 3 can be constituted by a personal computer, for example. A camera 4 is connected to the image arithmetic processing device 3.

  As will be described later, the image calculation processing device 3 detects the position of the workpiece W placed on the loading table 30 and supplies the detected position information to the robot control device 5. In addition to detecting the position of the workpiece W, the image arithmetic processing device 3 determines whether the workpiece W is arranged on the loading table 30 with the front and back sides being correct. The robot controller 5 controls the workpiece holding robot 20 based on the position information of the workpiece W if the workpiece W is arranged face up.

  In the configuration example shown in FIG. 3, the NC device 2 that controls the processing machine 10, the image arithmetic processing device 3 that constitutes the workpiece determination device, and the robot control device 5 that controls the workpiece holding robot 20 are provided separately. This is just an example.

  The NC device 2 may constitute a work determination device by providing the NC device 2 with the function of the image arithmetic processing device 3. The NC device 2 and the robot control device 5 may be integrated. The robot control device 5 may constitute a workpiece determination device by providing the robot control device 5 with the function of the image arithmetic processing device 3.

  In FIG. 3, the CAM 1 transfers CAD data to the NC device 2. The NC device 2 transfers the received CAD data to the image arithmetic processing device 3.

  A functional internal configuration of the image arithmetic processing device 3 will be described with reference to FIG. As illustrated in FIG. 4, the image calculation processing device 3 includes a template image creation unit 31, a template image filling unit 32, a template image enlargement / reduction / rotation processing unit 33, and a template image edge detection unit 34.

  Further, the image arithmetic processing device 3 includes a workpiece photographed image edge detection unit 35, a matching processing unit 36, and a front / back determination feature image extraction unit 38. The matching processing unit 36 includes a workpiece position detection unit 361 and a front / back determination unit 362.

  Each of the template image creation unit 31 to the matching processing unit 36 and the front / back determination feature image extraction unit 38 can be configured by a software module. At least a part of the template image creation unit 31 to the matching processing unit 36 and the front / back determination feature image extraction unit 38 may be configured by hardware. Software and hardware may be combined, and the proper use of both is arbitrary.

  4 will be described with reference to the flowcharts of FIGS. 5A to 5D. In FIG. 5A, the workpiece photographed image edge detection unit 35 acquires workpiece photographed image data output from the camera 4 in step S10. The work photographed image data is digital still image data obtained by photographing the work W placed on the loading table 30 by the camera 4.

  FIG. 6 conceptually shows the workpiece photographed image W30 indicated by the workpiece photographed image data. The workpiece photographed image W30 includes an image of the workpiece W arranged at the top on the loading table 30.

  The template image creation unit 31 acquires the CAD data transferred from the NC device 2 in step S21. In step S22, the template image creation unit 31 converts CAD data into image data to create a template image TW as shown in FIG. 7 (template image creation step). The template image TW includes a work line image ImW showing the work W. 7 to 12, 19, and 20, a rectangle indicated by an alternate long and short dash line indicates an outer edge of each image (image data).

  The CAD data is, for example, data in which coordinates indicating each point of the shape of the workpiece W are expressed in CSV format. The template image creation unit 31 creates a template image TW including a work line image ImW indicating the shape of the work W based on the CSV format data. The work line image ImW is an image showing the outline of the work W. Since the workpiece W has a cut-and-raised portion W0 in the plane as shown in FIG. 2, the workpiece line image ImW includes a cut-and-raised image ImW0 showing the cut-and-raised portion W0.

  At this time, the size of the workpiece line image ImW needs to be a size corresponding to the actual size of the workpiece W. Therefore, if the length per pixel is obtained in advance, the template image creation unit 31 can calculate how many pixels the dimension of the workpiece W should be expressed. The template image creation unit 31 draws the work line image ImW so as to have the calculated number of pixels.

  Specifically, if the length of one pixel corresponds to 0.5 mm and the length of a predetermined portion of the workpiece W is 10 cm, that portion may be expressed by 200 pixels.

  In step S23, the template image filling unit 32 fills the inside of the work line image ImW with, for example, white (filling template image creating step). FIG. 8 shows a template image TWw including a workpiece fill image ImWw in which the inside of the outline of the workpiece line image ImW is filled. The hatching in FIG. 8 indicates that the inside of the work line image ImW is filled. Hereinafter, the template image TWw will be referred to as a filled template image TWw.

  As shown in FIG. 8, the inside of the cut and raised image ImW0 should not be filled. The template image filling unit 32 may fill a portion other than the closed region existing inside the outline of the work line image ImW with a predetermined color. The reason for filling the inside of the work line image ImW to make the work filled image ImWw will be described later.

  Steps S21 to S23 in FIG. 5A show a fill template image creation process for creating a pattern matching template image (fill template image TWw) based on CAD data.

  By the way, instead of creating a pattern matching template image based on CAD data, an operator may previously create a pattern matching template image by photographing a workpiece with a camera. However, it is cumbersome for an operator to photograph a workpiece and create a template image for pattern matching. Therefore, it is preferable to create a template image for pattern matching based on CAD data.

  The front / back determination feature image extraction unit 38 is supplied with the filled template image TWw generated by the template image filling unit 32. In step S31, the front / back determination feature image extraction unit 38 creates the template back image ITWw shown in FIG. 9 by inverting the front and back of the filled template image TWw. The template back image ITWw includes an inverted work paint image ImIWw obtained by inverting the work line image ImW. The reverse workpiece filled image ImIWw includes the reverse cut-and-raised image ImIW0.

  In step S32, the front / back determination feature image extraction unit 38 matches the filled template image TWw shown in FIG. 8 with the template back image ITWw shown in FIG. When the filled template image TWw and the template back image ITWw are matched, the result is as shown in FIG. In step S33, the front / back determination feature image extraction unit 38 extracts overlapping edges (overlapping edges) EOL as shown in FIG.

  In step S34, the front / back determination feature image extraction unit 38 subtracts the overlapping edge EOL from the filled template image TWw to extract the front / back determination feature image FW shown in FIG. 12 (front / back determination feature image extraction step). The front / back determination feature image FW corresponds to an image obtained by cutting and raising only the image ImW0 from the filled template image TWw.

  Steps S31 to S34 in FIG. 5A show front / back determination feature image extraction processing for extracting the front / back determination feature image FW from the filled template image TWw.

  In step S41 of FIG. 5B, the template image enlargement / reduction / rotation processing unit 33 sets the minimum magnification for enlarging or reducing the filled template image TWw to min, the maximum magnification to max, and the magnification increment to k.

  When the work W on the loading table 30 is photographed by the camera 4, the size of the photographed image of the uppermost work W changes depending on the number of stacked works W. If the fill template image TWw is enlarged or reduced in increments of k in the range from the minimum magnification min to the maximum magnification max, the fill template image TWw is taken as the shot image of the workpiece W even if the size of the shot image of the workpiece W changes. It can be almost matched with the size of.

  The template image enlargement / reduction / rotation processing unit 33 sets the magnification b to min in step S42, and in step S43 multiplies the filled template image TWw by b (template image enlargement / reduction step).

  In step S44, the template image enlargement / reduction / rotation processing unit 33 initializes the angle i of the filled template image TWw to 0 °. For example, the angle at which the left and right sides of the fill template image TWw shown in FIG. 8 are parallel to the horizontal direction of the workpiece photographed image W30 may be set to 0 °.

  In step S45, the template image enlargement / reduction / rotation processing unit 33 rotates the filled template image TWw by an angle i (template image rotation step). However, since the angle i is 0 ° when the process proceeds from step S44 to step S45, the template image enlargement / reduction / rotation processing unit 33 does not substantially rotate the filled template image TWw.

  The template image edge detection unit 34 detects an edge of the filled template image TWw in step S51 of FIG. 5C (template image edge detection step). The template image edge detection unit 34 preferably uses Sobel transformation in order to detect the edge of the filled template image TWw.

  13A is a filter for detecting the horizontal edge of the filled template image TWw, and FIG. 13B is a filter for detecting the vertical edge of the filled template image TWw. . Each filter is constituted by a coefficient matrix of 3 rows and 3 columns.

  The template image edge detection unit 34 uses each of the filters shown in FIG. 13A for a total of nine pixels, including a target pixel and eight pixels in the upper, lower, left, right, and diagonal directions around the target pixel. Multiply by the corresponding position coefficient. The template image edge detection unit 34 adds the multiplication results for each of the nine pixels to detect the horizontal edge.

  Further, the template image edge detection unit 34 multiplies the nine pixels centered on the target pixel by a coefficient at a position corresponding to each of the filters in FIG. The template image edge detection unit 34 adds the multiplication results of each of the nine pixels to detect the vertical edge.

  The template image edge detection unit 34 detects the edge of the filled template image TWw by adding the detected horizontal edge and vertical edge.

  At this time, it is preferable that the template image edge detection unit 34 calculates the edge strength and the gradient direction for each pixel constituting the edge. In FIG. 14, each circle indicates a pixel. Assume that a horizontal edge value (edge intensity Eh) is detected as 5 and a vertical edge value (edge intensity Ev) is 1 as a result of Sobel transformation for a target pixel indicated by hatching.

  As shown in FIG. 14, the horizontal edge strength Eh and the vertical edge strength Ev can be considered as a horizontal vector Eh and a vertical vector Ev, respectively.

  The magnitude of the combined vector Ehv of the horizontal vector Eh and the vertical vector Ev indicates the edge intensity in each pixel, and the direction of the combined vector Ehv indicates the edge gradient direction. The edge strength Emagnitude can be expressed by Equation (1), and the edge gradient direction Edirection can be expressed by Equation (2).

  The template image edge detection unit 34 obtains the edge intensity Emagnitude and the edge gradient direction Edirection for all the pixels constituting the edge of the filled template image TWw.

  The template image edge detection unit 34 may obtain only the edge strength Emagnitude or may obtain only the edge gradient direction Edirection. The template image edge detection unit 34 preferably obtains at least the edge strength Emagnitude, and more preferably obtains both the edge strength Emagnitude and the edge gradient direction Edirection.

  Returning to FIG. 5C, in step S52, the workpiece photographed image edge detection unit 35 detects the edge of the workpiece photographed image W30 based on the workpiece photographed image data (work photographed image edge detection step). The workpiece photographed image edge detection unit 35 may perform noise reduction processing before detecting the edge of the workpiece photographed image W30.

  Similar to the template image edge detection unit 34, the workpiece photographed image edge detection unit 35 detects the edge of the workpiece photographed image W30 by Sobel transformation using the filters of FIGS. 13A and 13B. Similarly to the template image edge detection unit 34, the workpiece photographed image edge detection unit 35 may obtain the edge intensity Emagnitude and the edge gradient direction Edirection in all the pixels constituting the edge of the workpiece photographed image W30.

  The workpiece photographed image edge detection unit 35 may obtain only the edge intensity Emagnitude or may obtain only the edge gradient direction Edirection. The workpiece photographed image edge detection unit 35 preferably obtains at least the edge strength Emagnitude, and more preferably obtains both the edge strength Emagnitude and the edge gradient direction Edirection.

  In step S53, the matching processing unit 36 performs matching processing between the edge of the workpiece photographed image W30 and the edge of the filled template image TWw (matching processing step).

  Here, the reason why the template image filling unit 32 fills the inside of the work line image ImW to form the filled template image TWw will be described.

  FIG. 15A shows pixel values near the edge of the template image TW. In FIG. 15A, the left direction is the outside of the work line image ImW, and the right direction is the inside of the work line image ImW. When the template image TW is subjected to Sobel transformation, edges are detected in a state of two lines having an interval between each other.

  FIG. 15B shows pixel values in the vicinity of the edge of the filled template image TWw. Similarly, the left direction in FIG. 15B is the outside of the workpiece filled image ImWw, and the right direction is the inside of the workpiece filled image ImWw. By performing Sobel transform on the filled template image TWw, the edges become thick but have no gaps, that is, a substantially single line state, and the edges are appropriately detected.

  If the matching processing unit 36 performs a matching process between the edge of the workpiece photographed image W30 and the edge of the template image TW detected in a double line state where the edges are spaced, an error occurs and an appropriate matching process is performed. I will not. An appropriate matching process is performed by the matching processing unit 36 performing a matching process on the edge of the workpiece photographed image W30 and the edge of the filled template image TWw.

  Details of the matching processing by the matching processing unit 36 will be described with reference to FIG. In FIG. 16, ETW conceptually shows a template edge image obtained by Sobel conversion of the filled template image TWw by the template image edge detection unit 34. EW30 conceptually shows a workpiece shooting edge image obtained by performing Sobel transform on the workpiece shooting image W30.

  In FIG. 16, in order to facilitate understanding, the pixels constituting the respective edges of the template edge image ETW and the workpiece photographing edge image EW30 are conceptually shown enlarged.

  The matching processing unit 36 arranges the template edge image ETW at a predetermined position of the workpiece photographing edge image EW30, calculates the similarity between the two, and shifts the template edge image ETW one pixel at a time in the horizontal direction or the vertical direction to determine the similarity between the two. Calculate the degree.

  As shown in FIG. 16, when the pixel P1 of the template edge image ETW and the pixel P32 of the workpiece photographing edge image EW30 are at corresponding positions, the matching processing unit 36 calculates the degree of similarity between them. The matching processing unit 36 calculates the degree of similarity between the pixel of each edge of the template edge image ETW and the pixel of the workpiece photographing edge image EW30.

  For example, the matching processing unit 36 obtains a difference value between the values of the pixel of the template edge image ETW and the pixel at the corresponding position of the workpiece photographing edge image EW30, and calculates an integration value obtained by integrating the difference values for all the pixels as the template edge image. The overall similarity between the ETW and the workpiece photographing edge image EW30 can be set. In this case, the smaller the integral value, the higher the similarity.

  In the case of FIG. 16, since the positions of the edge of the workpiece photographing edge image EW30 and the edge of the template edge image ETW are shifted, the degree of similarity is relatively small.

  FIG. 17 shows a state in which the matching processing unit 36 shifts the template edge image ETW so that the pixel P1 of the template edge image ETW and the pixel P312 of the workpiece photographing edge image EW30 are in corresponding positions. In the case of FIG. 17, since the positions of the edge of the workpiece photographing edge image EW30 and the edge of the template edge image ETW are close to each other, the degree of similarity is relatively large.

  As described above, the matching processing unit 36 searches for the position of the template edge image ETW that is most similar to the template edge image ETW and the workpiece imaging edge image EW30 while shifting the template edge image ETW pixel by pixel in the horizontal direction or the vertical direction. To do.

  The matching processing unit 36 may calculate the degree of similarity using the horizontal edge strength Eh, the vertical edge strength Ev, and the edge strength Emagnitude described above. In the template edge image ETW, the horizontal edge strength Eh1, the vertical edge strength Ev1, the edge strength Emagnitude of the pixel in the workpiece photographing edge image EW30, the vertical edge strength Ev3, the edge. When the intensity Emagnitude is Em3, the similarity Ds of the pixel at the corresponding position can be obtained by Expression (3).

  Ds = (Eh1 × Eh3 + Ev1 × Ev3) / (Em1 × Em3) (3)

  The matching processing unit 36 obtains the similarity Ds with respect to the workpiece photographing edge image EW30 for all the pixels of the template edge image ETW. The matching processing unit 36 can set the integrated value obtained by integrating the similarity Ds obtained for all the pixels as the overall similarity between the template edge image ETW and the workpiece photographing edge image EW30.

  When Expression (3) is used, the similarity of the pixels increases as the pixels are similar, and the integrated value of the similarity Ds also increases. Hereinafter, a case where the similarity Ds is obtained using the expression (3) will be described as an example. The matching processing unit 36 may further calculate the similarity by comparing the edge gradient directions Edirection.

  Returning to FIG. 5C, in step S54, the matching processing unit 36 stores the highest similarity in the variable score. In step S55, the workpiece position detection unit 361 in the matching processing unit 36 detects the position of the template edge image ETW having the highest similarity to the workpiece imaging edge image EW30. Thus, the matching processing unit 36 (work position detection unit 361) can detect the position of the uppermost work W placed on the loading table 30.

  However, in step S55, the rough position of the workpiece W is merely detected by the template edge image ETW based on the selected template image TWw having the selected angle and size. The position of the workpiece W may not be detected accurately.

  Steps S51 to S55 in FIG. 5C show a workpiece imaging edge image position search process for searching for the position of the workpiece imaging edge image EW30 using the template edge image ETW based on the filled template image TWw of the selected angle and size. Yes.

  In step S61, the matching processing unit 36 determines whether it is the first matching process. If it is the first matching process (YES), that is, if it is a matching process using the template edge image ETW when the magnification b of the filled template image TWw is min and the angle i is 0 °, the matching processing unit 36 In step S62, the variable score is set to the maximum similarity Maxscore, and the process returns to step S46 in FIG. 5B.

  The template image enlargement / reduction / rotation processing unit 33 increases the angle i by 1 ° in step S46, and determines whether or not the filled template image TWw is rotated 360 ° in step S47. The increment of angle i is not limited to 1 °. The reason why the filled template image TWw is rotated is that the workpiece W on the loading table 30 may be arranged at various angles.

  If the fill template image TWw has not been rotated 360 ° (NO), the template image enlargement / reduction / rotation processing unit 33 rotates the fill template image TWw by an angle i in step S45. When the process proceeds from step S46 to step S45 through step S47, the filled template image TWw is rotated by an angle i.

  After the step S45, in the workpiece photographing edge image position search process of FIG. 5C, the highest similarity is stored in the variable score by the template edge image ETW based on the filled template image TWw with the angle i being 1 °. In the workpiece photographing edge image position search process, the position of the template edge image ETW having the highest similarity to the workpiece photographing edge image EW30 is detected.

  The matching processing unit 36 determines in step S61 that it is not the first matching process (NO), so in step S63, it determines whether Maxscore <score. If Maxscore <score (YES), the variable score obtained when the angle i is 1 ° is larger than the variable score obtained when the angle i is 0 °.

  In step S64, the matching processing unit 36 sets the variable score to the maximum similarity Maxscore, stores the magnification b, the angle i, and the detection position of the template edge image ETW at this time, and the processing is performed in step S46 of FIG. 5B. Return to.

  If Maxscore <score is not satisfied in step S63 (NO), the matching processing unit 36 returns the process to step S46 of FIG. 5B.

  In step S46, the template image enlargement / reduction / rotation processing unit 33 again increases the angle i by 1 °. If the fill template image TWw is not rotated by 360 ° in step S47 (NO), similarly, Steps S51 to S55 and S61 to S64 in FIG. 5C are repeated.

  By repeating steps S51 to S55 and S61 to S64 of FIG. 5C until the filled template image TWw is rotated 360 °, an angle i indicating the maximum similarity Maxscore is obtained from the variable scores at the respective angles i.

  If the filled template image TWw has been rotated 360 ° in step S47 (YES), the template image enlargement / reduction / rotation processing unit 33 increases the magnification b by k in step S48. In step S49, the template image enlargement / reduction / rotation processing unit 33 determines whether the magnification b is larger than the maximum magnification max.

  If the magnification b is not larger than the maximum magnification max (NO), the template image enlargement / reduction / rotation processing unit 33 repeats steps S44 and S45, and the matching processing unit 36 repeats steps S51 to S55 and S61 to S64. Further, the template image enlargement / reduction / rotation processing unit 33 repeats steps S46 to S49.

  If the magnification b is larger than the maximum magnification max (YES), steps S51 to S55 and S61 to S64 are repeated until the fill template image TWw is rotated 360 ° at all magnifications b up to the maximum magnification max. That is.

  As described above, the workpiece position detection unit 361 selects one set of size and angle from the plurality of sizes and the plurality of angles of the template edge image ETW, and the position of the template edge image ETW with respect to the workpiece photographing edge image EW30. The template edge image ETW and the workpiece photographing edge image EW30 are matched while shifting. As a result of matching, the workpiece position detection unit 361 preliminarily detects the position of the workpiece W from the position of the template edge image ETW having the highest similarity between the template edge image ETW and the workpiece photographing edge image EW30.

  The workpiece position detecting unit 361 sequentially changes the size and angle pairs, and similarly matches the template edge image ETW and the workpiece imaging edge image EW30 with each of the plurality of size and angle pairs, thereby obtaining the template edge image ETW. The position of the workpiece W is preliminarily detected from the position of the template edge image ETW having the highest degree of similarity between the workpiece photographing edge image EW30.

  The workpiece position detection unit 361 detects the position of the template edge image ETW in the group having the highest similarity among the plurality of groups as the position of the workpiece W finally placed on the loading table 30 (work position detection). Process).

  Taking the case where Expression (3) is used as an example, the workpiece position detection unit 361 obtains the largest integral value among the integral values of the similarity Ds calculated by all sets of template edge images ETW. The position of the template edge image ETW may be determined as the position of the workpiece W.

  As described above, in step S64, among the combinations of all the magnifications b and all the angles i, the magnification b, the angle i and the detection position of the template edge image ETW when the maximum similarity Maxscore is obtained are stored. Will be.

  Steps S41 to S43, S48, and S49 in FIG. 5B show a filling template image enlarging / reducing process for enlarging or reducing the filling template image TWw for each increment k in the range from the minimum magnification min to the maximum magnification max. Steps S61 to S64 in FIG. 5C show the maximum similarity condition storing process for detecting and storing the magnification b, the angle i, and the detection position of the template edge image ETW when the maximum similarity Maxscore is shown.

  If the magnification b is larger than the maximum magnification max in step S49 (YES), the front / back determination feature image extraction unit 38 in step S71 in FIG. It is determined whether or not the number of pixels of the front / back determination feature image FW extracted in S34 is greater than zero.

  If the number of pixels of the front / back determination feature image FW is greater than 0 (YES), the front / back determination unit 362 calculates the magnification b, the angle i, and the template edge obtained by the matching process having the maximum similarity Maxscore in step S72. The detection position of the image ETW is read out. In step S73, the front / back determination unit 362 enlarges or reduces the front / back determination feature image FW at the magnification b, rotates the feature image FW for front / back determination by the angle i, and moves the front / back determination feature image FW to the detection position of the template edge image ETW. .

  The front / back determination feature image FW is enlarged or reduced by the magnification b when the maximum similarity Maxscore is obtained, and rotated by the angle i, whereby the front / back determination feature image FW is the size of the workpiece W in the workpiece photographed image W30. The size and angle correspond to the height and angle. The detection position of the template edge image ETW indicates the position of the work W in the work photographed image W30 (the position of the work W on the loading table 30). Therefore, if the front / back determination feature image FW is moved to the detection position of the template edge image ETW, the front / back determination feature image FW moves to a position corresponding to the position of the workpiece W.

  The front / back determination unit 362 calculates the similarity between the workpiece photographing edge image EW30 and the front / back determination feature image FW in step S74, and determines whether the similarity is equal to or greater than a predetermined threshold value in step S75. (Front / back determination step). Also here, it is assumed that the value indicating the similarity increases as the workpiece photographing edge image EW30 and the front / back determination feature image FW become similar.

  If the similarity is equal to or greater than the threshold (YES), it means that the workpiece W is arranged on the loading table 30 with the front and back being correct, and the matching processing unit 36 moves the process to step S81. If it is less than the threshold value (NO), it means that the work W is not arranged in the correct state on the loading table 30, and the matching processing unit 36 (front / back determination unit 362) shifts the processing to step S82.

  If the number of pixels of the front / back determination feature image FW is not larger than 0 in step S71 (NO), the front / back determination unit 362 determines that front / back determination is impossible in step S76, and the process proceeds to step S81. . The fact that the number of pixels of the front / back determination feature image FW is not greater than 0 (the number of pixels is 0) indicates that the feature image for determining the front and back of the workpiece W could not be extracted in step S34 of FIG. 5A. That's what it means. If there is no difference in shape between the front-facing work W and the back-facing work W, the number of pixels of the front-back determination feature image FW is zero.

  Steps S71 to S76 in FIG. 5D indicate front / back determination processing for determining whether or not the workpiece W placed on the loading table 30 is correctly placed face up.

  In step S <b> 81, the matching processing unit 36 transfers the position information (work position information) of the work W to the robot control device 5 and ends the process. In step S 82, the matching processing unit 36 (front / back determination unit 362) transmits error information indicating that the workpiece W is not correctly placed face-up to the NC device 2 and ends the process.

  The NC device 2 that has received the error information may cause the display / operation unit 21 to display that the workpiece W is placed face down, or perform robot control so that the workpiece holding robot 20 does not hold the workpiece W. The device 5 may be controlled.

  The processing of steps S73 to S75 will be specifically described with reference to FIGS. FIG. 18A shows the workpiece photographing edge image EW30 when the workpiece W is correctly arranged on the loading table 30. In FIGS. 18A and 18B, only the workpiece photographing edge image EW30 of the uppermost workpiece W is shown for simplification. The workpiece photographing edge image EW30 includes an edge EW of the outer periphery of the workpiece W and an edge EW0 of the cut and raised portion W0.

  The workpiece W is counterclockwise when the corner P0 (see FIG. 1) of the loading table 30 is the origin (0, 0) and the lower left corner of the workpiece W is located at a predetermined coordinate (x, y). It is assumed that it is rotated by 30 °.

  FIG. 18B shows the workpiece photographing edge image EW30 when the workpiece W is mistakenly placed on the loading table 30 in the reverse direction. The workpiece photographing edge image EW30 includes an edge EW of the outer periphery of the workpiece W and an edge EW0 of the cut and raised portion W0. The edge EW0 here is an edge when the cut and raised portion W0 is viewed from the back surface of the workpiece W. Similarly, it is assumed that the workpiece W is rotated by 30 ° counterclockwise with the lower left corner located at the coordinate position (x, y).

  In the case of FIGS. 18A and 18B, the angle i when the maximum similarity Maxscore is obtained is 30 °. The magnification b when the maximum similarity Maxscore is obtained is 1.2. As shown in FIG. 19, the front / back determination unit 362 multiplies the front / back determination feature image FW by 1.2 and rotates it 30 ° counterclockwise to obtain a front / back determination feature image FW ′. The cut-and-raised image ImW0 included in the front-and-back determination feature image FW is also 1.2 times, rotated 30 ° counterclockwise to become a cut-and-raised image ImW0 '.

  When the front / back determination feature image FW ′ shown in FIG. 19 is superimposed on FIG. 18 (a), the state shown in FIG. 20 (a) is obtained. The edge EW0 of the cut-and-raised portion W0 and the cut-and-raised image ImW0 'may not completely overlap. However, for convenience, it is assumed here that the edge EW0 and the cut-and-raised image ImW0' completely overlap.

  When the front / back determination unit 362 calculates the similarity between the workpiece photographing edge image EW30 and the front / back determination feature image FW ′, the edge EW0 and the cut-out image ImW0 ′ overlap with each other, and the similarity is a relatively large value. It becomes a predetermined threshold value or more. Therefore, the front / back determination unit 362 can determine that the workpiece W is arranged face up.

  When the front / back determination feature image FW ′ shown in FIG. 19 is superimposed on FIG. 18B, a state as shown in FIG. 20B is obtained. When the front / back determination unit 362 calculates the similarity between the workpiece photographing edge image EW30 and the front / back determination feature image FW ′, the edge EW0 and the cut-out image ImW0 ′ are completely displaced, and thus the similarity is extremely small. Value, less than the threshold. Therefore, the front / back determination unit 362 can determine that the workpiece W is arranged face down.

  In FIG. 4, the robot coordinate conversion unit 51 in the robot control device 5 converts the input workpiece position information into robot coordinates that determine how to move the workpiece holding robot 20. The robot coordinates are at least one of a rotation angle of the workpiece holding robot 20, a position on the guide rail 201, and a joint angle.

  The robot control device 5 moves the workpiece holding robot 20 based on the robot coordinates to hold the uppermost workpiece W on the loading table 30. The workpiece holding robot 20 conveys the workpiece W to the processing machine 10 and unloads the processed workpiece W to a predetermined position.

  As described above, since the origin position of the workpiece holding robot 20 coincides with the origin of the photographed image, the position of the workpiece W detected as described above is the position where the workpiece holding robot 20 holds the workpiece W as it is. Become.

  As described above, the workpiece determination apparatus according to this embodiment includes the front / back determination feature image extraction unit 38 and the front / back determination unit 362. The front / back determination feature image extraction unit 38 inverts the front and back of the template image (filled template image TWw) to generate a template back image ITWw indicating the back side of the workpiece. The front / back determination feature image extraction unit 38 extracts a front / back determination feature image FW for determining the front / back of the workpiece W by subtracting the overlapping edge EOL between the template image and the template back image ITWw from the template image.

  The front / back determination unit 362 moves the front / back determination feature image FW (or FW ′) to the position of the workpiece W detected by the workpiece position detection unit 361 and resembles the workpiece photographing edge image EW30 and the front / back determination feature image FW. The degree is calculated, and the front and back of the workpiece W are determined based on the degree of similarity.

  Therefore, according to the workpiece determination apparatus of the present embodiment, since it is not necessary to pattern match the workpiece photographed image W30 and the front and back template images, without significantly increasing the processing time required for pattern matching. The front and back of the workpiece W can be determined efficiently and accurately.

  The workpiece determination method according to the present embodiment includes a front / back determination feature image extraction step for extracting a front / back determination feature image FW for determining the front / back of the workpiece W, a workpiece imaging edge image EW30, and a front / back determination feature image FW (or And a front / back determination step of determining the front / back of the workpiece W based on the similarity to FW ′). According to the workpiece determination method of this embodiment, the front and back of the workpiece W can be determined efficiently and accurately.

  The workpiece determination apparatus according to the present embodiment includes a template image enlargement / reduction / rotation processing unit 33 that enlarges or reduces a template image to a plurality of sizes within a predetermined size range and rotates the template image to a plurality of angles. Is preferred. The workpiece position detection method of the present embodiment preferably includes a template image enlargement / reduction / rotation process.

  If comprised in this way, even if the magnitude | size of the picked-up image of the uppermost workpiece | work W changes with the number of the workpiece | work W piled up, even if the workpiece | work W is arrange | positioned at arbitrary angles, the position of the workpiece | work W Can be accurately detected.

  The workpiece determination apparatus and method according to the present embodiment enlarges or reduces the front / back determination feature image FW so as to correspond to the size of the enlarged or reduced template image, and rotates the feature image FW so as to correspond to the angle of the template image. Therefore, according to the workpiece determination apparatus and method of the present embodiment, the degree of similarity between the workpiece imaging edge image EW30 and the front / back determination feature image FW can be obtained regardless of whether the template image is enlarged or reduced, or the template image is rotated. It can be calculated accurately and the front and back of the workpiece W can be determined.

  By the way, as a template image for pattern matching with the workpiece W arranged at the placement place, a template image that shows the outline of the workpiece W created based on CAD data indicating the shape of the workpiece W as a line image. It is preferable to use a filled template image TWw in which the inside is filled.

  If the filled template image TWw is used, a complicated operation of photographing the workpiece W and creating a template image becomes unnecessary. When photographing the workpiece W, a work table dedicated to photographing is generally used. If the filled template image TWw is used, a work table dedicated to photographing is not necessary, and it is not necessary to recreate the template image even if the environment changes. If the fill template image TWw is used, the edge is appropriately detected, so that the edge of the workpiece photographed image W30 and the edge of the fill template image TWw are accurately matched.

  The present invention is not limited to the embodiment described above, and various modifications can be made without departing from the scope of the present invention.

1 CAM
2 NC device 3 Image processing unit 4 Camera 5 Robot control device 10 Processing machine 20 Workpiece holding robot 30 Loading table (placement location)
31 Template image creation unit 32 Template image filling unit 33 Template image enlargement / reduction / rotation processing unit 34 Template image edge detection unit 35 Work photographed image edge detection unit 36 Matching processing unit 38 Front / back determination feature image extraction unit 361 Work position detection unit 362 Front / back judgment part W Workpiece

Claims (8)

A template image edge detection unit that detects the edge of the template image for pattern matching with the workpiece placed at the placement location, and generates a template edge image;
A workpiece photographed image edge detection unit that detects an edge of the workpiece photographed image data obtained by photographing the workpiece arranged in the above-described place and generates a workpiece photographed edge image;
A workpiece position detection unit that matches the template edge image and the workpiece photographing edge image to detect the position of the workpiece arranged at the above-described placement location;
A template back image showing the back side of the workpiece is generated by inverting the front and back sides of the template image, and the front and back sides of the workpiece are determined by subtracting an edge where the template image and the template back image overlap from the template image. A front / back determination feature image extraction unit for extracting a front / back determination feature image for performing,
The front / back determination feature image is moved to the position of the workpiece detected by the workpiece position detection unit, the similarity between the workpiece photographing edge image and the front / back determination feature image is calculated, and based on the similarity A front / back determination unit for determining the front / back of the workpiece arranged at the placement location;
A workpiece determination apparatus comprising: A template image enlargement / reduction / rotation processing unit that enlarges or reduces the template image to a plurality of sizes within a predetermined size range and rotates the template image at a plurality of angles;
The template image edge detection unit detects edges of the template image at each of a plurality of sizes and a plurality of angles of the template image, and generates a plurality of template edge images,
The workpiece position detection unit is configured to shift the position of the template edge image with respect to the workpiece photographing edge image while shifting the position of the template edge image with respect to each of a plurality of sizes and a plurality of sets of sizes and angles at a plurality of angles of the template edge image. Is the position of the template edge image having the highest similarity between the template edge image and the workpiece photographing edge image, and the highest similarity among the plurality of sets. The work determination apparatus according to claim 1, wherein the position of the template edge image in the set is detected as the position of the work placed at the placement location.   The front / back determination unit enlarges or reduces the front / back determination feature image so as to correspond to the size of the template edge image of the set with the highest similarity, and the template edge of the set with the highest similarity. 3. The feature image for front / back determination that has been rotated by an angle of an image, and that has been enlarged or reduced and rotated, is moved to the position of the template edge image of the set having the highest degree of similarity. Work determination device. A template image creation unit that creates a template image indicating a contour line of the workpiece as a line image based on CAD data indicating the shape of the workpiece;
A template image filling unit that fills the inside of the outline with a predetermined color and creates a filled template image;
Further comprising
The said template image edge detection part detects the edge of the said fill template image, and uses the edge of the said fill template image as the said template edge image. The Claim 1 characterized by the above-mentioned. Work determination device. A template image edge detection step of detecting a template image edge for pattern matching with a workpiece placed at a placement location and generating a template edge image;
A workpiece photographed image edge detection step of detecting an edge of workpiece photographed image data obtained by photographing the workpiece arranged at the above-mentioned placement location, and generating a workpiece photographed edge image;
Matching the template edge image and the workpiece photographing edge image to detect the position of the workpiece arranged at the placement location, a workpiece position detection step;
A template back image showing the back side of the workpiece is generated by inverting the front and back sides of the template image, and the front and back sides of the workpiece are determined by subtracting an edge where the template image and the template back image overlap from the template image. A feature image extracting step for front / back determination for extracting a front / back determination feature image for performing,
Based on the similarity, the feature image for front / back determination is moved to the position of the workpiece detected in the workpiece position detection step, and the similarity between the workpiece photographing edge image and the feature image for front / back determination is calculated. Front and back determination step for determining the front and back of the work placed at the placement location,
The work determination method characterized by including. The template image further includes a template image enlargement / reduction / rotation step of enlarging or reducing the template image to a plurality of sizes within a predetermined size range and rotating the template image to a plurality of angles,
In the template image edge detection step, an edge of the template image is detected at each of a plurality of sizes and a plurality of angles of the template image to generate a plurality of template edge images,
In the workpiece position detection step, the template edge image is shifted while shifting the position of the template edge image with respect to the workpiece photographing edge image for each of a plurality of sizes of the template edge image and a plurality of sets of sizes and angles at a plurality of angles. When the image and the workpiece photographing edge image are matched, the template edge image has the highest similarity between the template edge image and the workpiece photographing edge image, and the similarity is the highest among the plurality of sets. The work determination method according to claim 5, wherein the position of the template edge image in a high set is detected as the position of the work placed at the placement location.   In the front / back determination step, the front / back determination feature image is enlarged or reduced so as to correspond to the size of the template edge image having the highest similarity, and the template having the highest similarity is set. The feature image image for front / back determination that has been rotated by an angle of an edge image and enlarged / reduced / rotated is moved to the position of the template edge image of the set having the highest similarity. Work determination method. Based on CAD data indicating the shape of the workpiece, a template image creating step for creating a template image showing the outline of the workpiece as a line image;
A fill template image creating step of filling the inside of the outline with a predetermined color and creating a fill template image;
Further including
The edge of the said filling template image is detected in the said template image edge detection process, The edge of the said filling template image is used as the said template edge image. The any one of Claims 5-7 characterized by the above-mentioned. Work determination method.

Images