JP2010210376A - Template matching system and template matching method - Google Patents

Abstract

It is determined based on a template acquired by template matching whether a posture in which a part is placed is stable or not, and if unstable, a warning is generated.
A template matching unit 100 extracts a shape based on acquired design data of a workpiece 130, generates a plurality of projection images projected onto a plane in a plurality of postures, and shapes the postures according to the respective postures. Based on the result of restoring the shape posture, calculate the stability when it is placed for each posture, and associate it with the template and stability generated for each projection image A template corresponding to the captured image of the photographed workpiece 130 is selected by matching, and it is determined whether or not to warn about the posture on which the workpiece 130 is placed based on the stability corresponding to the selected template. If it is determined that a warning is generated, a warning signal indicating a warning is generated.
[Selection] Figure 2

Description

  The present invention relates to a template matching system and a template matching method.

  In a production line in which the assembly process is automated, in order to accurately grasp and assemble components, it is necessary to accurately know the position of the components as well as the position of the components to be supplied. In this case, as shown in Patent Document 1 below, a template of a part serving as a reference pattern is created corresponding to various postures, and an image of the supplied part is compared with various templates, and the matching is achieved. By determining the degree and obtaining the template having the highest degree of coincidence, information on the posture on which the part is placed was obtained.

JP 2008-158626 A

By the way, when a part is conveyed by a belt conveyance apparatus etc. and is carried to a predetermined position, various aspects can be assumed for the posture of the part placed on the belt conveyance apparatus. For example, when components are placed on the belt transport device in a state where they are overlapped with each other, the individual components are in an unstable posture or there is a risk of breakage due to the fragile part of the component being on the bottom side. I did. Therefore, depending on the posture in which the components are placed, it is necessary to stop the belt conveying device, correct the posture of the components, and rearrange the overlapping components.
However, since the template acquired by the above method does not have information on the stability of the posture where the component is placed, the posture where the component is placed can be obtained from the template, but the posture is stable. I couldn't know if it was a posture or an unstable posture.

  SUMMARY An advantage of some aspects of the invention is to solve at least a part of the problems described above, and the invention can be implemented as the following forms or application examples.

[Application Example 1]
The template matching system according to this application example includes an acquisition unit that acquires design data of a target, an extraction unit that extracts the shape of the target based on the acquired design data, and the extracted shape using a plurality of postures. Projecting means for generating a plurality of projected images projected onto a plane in step, and the target of the shape is virtually changed according to each of the plurality of postures, and the target of the shape is restored based on the result of the restoration of the posture of the shape Calculation means for calculating the stability when an object is placed for each of the plurality of postures, template generation means for generating a template corresponding to each of the plurality of projection images, the generated template and the template Storage means for storing the corresponding stability, photographing means for photographing the placed target and outputting a photographed image, and storage means. Based on the matching means for selecting the template corresponding to the photographed image by comparing the template and the photographed image and determining the degree of coincidence, and the stability stored in accordance with the selected template A warning determination means for determining whether or not to warn about the posture on which the target is placed, and a warning signal indicating a warning when it is determined to warn about the posture on which the target is placed. And a warning signal generation means.

  According to such a configuration, the shape is extracted based on the design data of the target object, and a plurality of projection images obtained by projecting the extracted shape in a plurality of postures are generated, and the shape is virtual according to each posture. The stability when the target is placed on a plane for each of a plurality of postures is calculated based on the result of the change in the shape and the posture of the shape is restored, and a template corresponding to each of the plurality of projected images is calculated. Generated and the generated template is stored with the stability. Furthermore, the captured image obtained by photographing the placed target is compared with the stored template to determine the degree of coincidence, so that the corresponding template is selected, and the stability according to the selected template is achieved. Based on this, it is determined whether or not to warn about the posture on which the target is placed. If it is determined to warn, a warning signal is generated. Accordingly, since template matching is performed using a template having stability information, a template corresponding to the posture of the target is selected, and a warning signal can be generated according to the stability of the posture of the target. .

[Application Example 2]
In the template matching system according to the application example, the calculation unit calculates a contact point of the shape that first contacts the plane when the shape is brought close to the plane in each of the plurality of postures. Using the calculation means and the contact point calculated in each of the plurality of postures as fulcrums, the posture of the shape placed on the plane is virtually minutely changed, and then the posture of the shape returns to the original. It is preferable to include stability determination means for determining stability when the target is placed for each of the plurality of postures depending on whether or not the target is placed.

  According to such a configuration, the stability of each posture of the target can be calculated dynamically.

[Application Example 3]
In the template matching system according to the application example, the warning determination unit warns about the posture on which the target is placed based on information on the target in addition to the stability according to the template, It is preferable to determine whether or not.

  According to such a configuration, a warning signal can be generated in accordance with information related to the target in addition to the stability of the posture.

[Application Example 4]
In the template matching system according to the application example, it is preferable that the warning signal displays information indicating a warning on a display unit.

  According to such a configuration, a warning can be notified by the display means regarding the posture on which the target is placed.

[Application Example 5]
The template matching system according to the application example instructs the posture changing unit capable of gripping the target to change the posture of the target when it is determined not to warn about the posture on which the target is placed. You may further provide an attitude | position change instruction | indication means.

[Application Example 6]
The template matching method according to this application example includes a step of acquiring design data of a target, a step of extracting the shape of the target based on the acquired design data, and the extracted shape in a plurality of postures. Generating a plurality of projected images projected onto the image, and changing the posture of the shape virtually according to each of the plurality of postures, and based on a result of restoring the posture of the shape, the target is the A step of calculating the stability when mounted for each of a plurality of postures; a step of generating a template corresponding to each of the plurality of projection images; and the generated template and the stability corresponding to the template. The step of storing, the step of photographing the placed target and outputting the photographed image, and the stored template and the photographed image are compared to determine the degree of coincidence Whether to warn about the posture on which the target is placed based on the step of selecting the template corresponding to the captured image and the stability stored according to the selected template. And a step of generating a warning signal indicating a warning when it is determined to warn about the posture on which the target is placed.

  According to such a method, a shape is extracted based on the design data of the target object, and a plurality of projected images obtained by projecting the extracted shape in a plurality of postures are generated, and the shape is virtual according to each posture. The stability when the target is placed on a plane for each of a plurality of postures is calculated based on the result of the change in the shape and the posture of the shape is restored, and a template corresponding to each of the plurality of projected images is calculated. Generated and the generated template is stored with the stability. Furthermore, the captured image obtained by photographing the placed target is compared with the stored template to determine the degree of coincidence, so that the corresponding template is selected, and the stability according to the selected template is achieved. Based on this, it is determined whether or not to warn about the posture on which the target is placed. If it is determined to warn, a warning signal is generated. Accordingly, since template matching is performed using a template having stability information, a template corresponding to the posture of the target is selected, and a warning signal can be generated according to the stability of the posture of the target. .

The figure which shows the external appearance of the workpiece | work attitude | position change apparatus which concerns on this embodiment. The figure which shows the function structure of the workpiece | work attitude | position change apparatus which concerns on this embodiment. The figure explaining the stability calculation of a workpiece | work. The flowchart which shows the flow of the process by the workpiece | work attitude | position change apparatus which concerns on this embodiment.

  Hereinafter, a workpiece posture changing apparatus to which a template generation system apparatus is applied will be described with reference to the drawings.

(Embodiment)
FIG. 1 is a diagram showing an overview of the workpiece posture changing device 5. The workpiece posture changing device 5 includes a robot 10 and a control device 50 that controls the robot 10 to change the workpiece posture, and a placement posture of a workpiece 130 that is a target object conveyed in a certain direction by a belt conveying device 135. Is detected by pattern matching, and if necessary, the robot 10 holds the workpiece 130, changes the workpiece 130 to a predetermined posture, and places it on the floor surface.
The robot 10 is posture changing means for changing the posture of the target. The robot base 25 is a base installed on the installation surface, the first shaft 11 that rotates on the vertical axis with respect to the installation surface, and the arm 17A on the horizontal axis. The second shaft 12 that rotates, the third shaft 13 that rotates in the axial direction of the arm 17A, the fourth shaft 14 that is fixed to the arm 17A and rotates the arm 17B in the horizontal direction, and the fifth shaft that rotates in the axial direction of the arm 17B 15 and a six-axis control multi-joint industrial robot that is fixed to the arm 17B and uses the sixth axis 16 that rotates the wrist 18 in the horizontal direction as moving parts. A hand 19 is attached to the tip of the wrist 18. The hand 19 can grasp a supplied component, change it to a predetermined posture, convey it to a predetermined position, and perform an assembling operation or the like.

These drive shafts are configured to rotate by driving a plurality of actuators 30 (FIG. 2) that are operated by a motor, a pneumatic device, or the like (not shown). It is configured to drive based on a control signal sent via the cable 85.
In the present embodiment, the six-axis control articulated industrial robot is adopted as a means for changing and assembling the parts. However, the present invention is not limited to this, and a scalar type robot may be used. Also, a dedicated machine such as a pick-and-place unit may be used in which driving means for driving in one direction are combined in multiple stages.

The control device 50 includes a computer 80, a display 82, a keyboard 84, the digital camera 20, and an indicator lamp 160. The computer 80 includes hardware resources such as a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), a HDD (Hard Disk Drive), a sequencer, a robot controller, and a drive unit. Various functions stored in ROM, HDD, and the like cooperate with each other in an organic manner to realize the functions of the functional units described later.
In addition, the digital camera 20 as an imaging unit is installed at a position where the workpiece 130 can be imaged, and the imaging device such as a CCD (Charge Coupled Device) (not shown) captures an image under imaging conditions suitable for the workpiece 130. The processed image signal is configured to be sent to the computer 80 via the cable 85. In this embodiment, the digital camera 20 is installed separately from the robot 10, but may be installed on the wrist 18 of the robot 10. Further, the viewpoint of the digital camera 20 may be configured to be freely changed according to an instruction from the control device 50. The indicator lamp 160 is lit in red, yellow, and green according to an instruction from the computer 80. The indicator lamp 160 is not limited to such a mode, and may be a red or other rotating lamp.

FIG. 2 is a diagram illustrating a functional configuration of the workpiece posture changing device 5. The control device 50 includes a template matching unit 100, a robot control unit 126, a driver 124, a work information storage unit 129, an indicator lamp control unit 132, and a UI display control unit 134. The template matching unit 100 is a template matching system including a template acquisition unit 110, a matching unit 105, a warning determination unit 106, an attitude correction instruction unit 108, and a warning signal generation unit 109.
Based on CAD (Computer Aided Design) data of the work 130, the template acquisition unit 110 acquires all templates that may be taken when the work 130 is projected two-dimensionally. The template acquisition unit 110 includes a CAD data acquisition unit 112, a shape extraction unit 114, a projection unit 116, a template generation unit 118, a stability calculation unit 120, and a template storage unit 122.
The CAD data acquisition unit 112 is an acquisition unit that acquires design data (CAD data) of the work 130 from the CAD database 128. The CAD database 128 may be stored in the computer 80 of the control device 50, or may be stored in an external server and acquired via communication. The CAD data acquired by the CAD data acquisition unit 112 is sent to the shape extraction unit 114.

The shape extraction unit 114 is an extraction unit that analyzes the CAD data sent from the CAD data acquisition unit 112 and extracts a CAD model 150 (FIG. 3) having a three-dimensional shape related to the workpiece 130. In this embodiment, CAD data is assumed to be data in which the work 130 is described in IGES (Initial Graphics Exchange Specification), VRML (Virtual Reality Modeling Language), or the like. Further, the present invention is not limited to analysis of CAD data, and a three-dimensional shape may be extracted through an application that operates CAD data. Information about the CAD model 150 extracted by the shape extraction unit 114 is sent to the stability calculation unit 120 and the projection unit 116.
The stability calculation unit 120 includes a contact point calculation unit 140 and a stability determination unit 145, and calculates the stability for each posture on which the workpiece 130 is placed as a calculation unit. Details of the function of the stability calculation unit 120 will be described with reference to FIG.

The contact point calculation unit 140 is a contact point calculation unit that uses the CAD model 150 and assumes all possible postures of the work 130 and calculates a contact point in contact with the floor surface. Specifically, the CAD model 150 extracted by the shape extraction unit 114 is rotated in various three-dimensional directions. At this time, the rotating CAD model 150 is translated and brought close to a flat floor surface to calculate a contact point. Here, when the surface shape of the workpiece 130 is composed of only a plane, or when the CAD model 150 of the workpiece 130 is described in VRML and expressed by approximation with a triangular polygon, it is shown in FIG. As described above, since the distance (h ₁ , h ₂ ) between each vertex of the CAD model 150 and the plane may be calculated, the contact point can be calculated easily.
As shown in FIGS. 3B and 3C, the stability determination unit 145 gives a minute change by virtually shifting the CAD model 150 of the workpiece 130 by a minute angle with the calculated contact points as fulcrums. This is a stability determination means for determining for each contact point whether or not to restore the original state. Specifically, an axis (not shown) passing through the fulcrum is set, and a moment due to gravity is calculated around this axis, and it is determined whether or not to return to the original position. Further, a change in the magnitude of the moment when a minute change is given to the CAD model 150 may be acquired, and determination may be made based on the stability of this change. The stability calculation unit 120 holds the stability for each fulcrum, that is, for each posture on which the work 130 is placed. Information regarding the stability for each posture of the workpiece 130 is referred to when the template generated by the template generation unit 118 is stored.

The projecting unit 116 is a projecting unit that projects a captured image based on internal parameters of the digital camera 20 acquired in advance and external parameters indicating a shooting direction and the like. That is, based on the external parameters, the camera coordinates of the digital camera 20 can be converted from a three-dimensional shape of various postures of the workpiece 130 that can be calculated from the CAD model 150 by affine transformation, which is a known transformation method for enlarging, reducing, or rotating an image. Project into the system. Further, the image is projected onto the image plane of the digital camera 20 based on the internal parameters. When the posture of the workpiece 130 is determined, the workpiece 130 is projected onto the image plane of the digital camera 20. The projected image projected here is sent to the template generation unit 118.
The template generation unit 118 is a generation unit that generates a template from the projection image sent from the projection unit 116 and stores the generated template information in the template storage unit 122. The template generation unit 118 reads information relating to the stability corresponding to the generated template from the stability calculation unit 120 and associates the information and sends the information to the template storage unit 122. The template storage unit 122 is a storage unit, and stores the template generated by the template generation unit 118 and information related to the stability associated with the template in association with each other.

  The matching unit 105 is a matching unit that performs matching between a photographed image of the work 130 photographed by the digital camera 20 and a template projected onto an image plane stored in the template storage unit 122. In this case, you may match with the template which projected the picked-up image in parallel. The matching method is not limited, and it may be determined that the posture is detected if an edge image is generated from the photographed image, the shapes are matched, and a matching is performed exceeding a preset threshold value. The template is preferably projected onto the image plane. However, if the posture of the photographed work 130 in the world coordinate system can be obtained, the photographed image may be converted into the world coordinate system for matching. The posture information detected by the matching unit 105 is sent to the warning determination unit 106.

The warning determination unit 106 acquires information about the stability of the matched template from the template storage unit 122 from the posture information detected by the matching unit 105, and further acquires information about the work 130 from the work information storage unit 129. These are warning determination means for determining whether or not to issue a warning to the user based on these pieces of information. For example, when the posture of the workpiece 130 detected by the matching unit 105 is unstable, or an area where the workpiece 130 is to be held based on information stored in the workpiece information storage unit 129 is a holding prohibited portion of the workpiece 130. If there is, the warning determination unit 106 determines transmission of a warning to the user.
In addition, the holding | maintenance prohibition part of the workpiece | work 130 is destroyed when the electronic circuit and the precision component are mounted in the inside of the workpiece | work 130, or surface vicinity, and the hand 19 of the robot 10 hold | grips, or precision deteriorates. This is a highly likely area. When the warning determination unit 106 determines the transmission of a warning to the user, the warning signal generation unit 109 is instructed to generate a warning signal. On the other hand, when the warning determination unit 106 determines that it is not necessary to send a warning to the user, the posture correction instruction unit 108 is instructed to correct the posture of the workpiece 130.

  The warning signal generation unit 109 is a warning signal generation unit that displays a warning that the posture of the workpiece 130 is defective on the display unit in response to an instruction from the warning determination unit 106 and notifies the user. Specifically, the warning signal generation unit 109 generates a warning signal for instructing display to the UI (User Interface) display control unit 134 and the indicator lamp control unit 132. As a result, the UI display control unit 134 displays a message box (not shown) indicating a warning on the display 82 and the indicator light control unit 132 turns on the lamp. As a result, the user visually recognizes these warnings, temporarily pauses the belt conveyance device 135, corrects the posture of the workpiece 130, or removes the workpiece 130 with a poor posture, and then releases the pause of the belt conveyance device 135. . Note that the warning to the user is not limited to the above-described means, and a method of warning the user visually and, for example, a warning by voice or the like may be used together.

The posture correction instruction unit 108 instructs the robot control unit 126 to drive the robot 10 in order to change the workpiece 130 to a predetermined posture based on information on the detected posture in response to an instruction from the warning determination unit 106. Change instruction means.
The robot control unit 126 and the driver 124 are control means for controlling the operation of the operation unit such as the arm 17. The robot control unit 126 calculates drive amounts of the plurality of actuators 30 for moving the robot 10 based on an instruction to change the posture of the workpiece 130, and sends drive information regarding the calculated drive amounts to the driver 124. The driver 124 generates a drive signal for each actuator 30 based on the drive information sent from the robot controller 126 and sends the drive signal to each actuator 30. As a result, the arm 17 moves to a predetermined position, and the hand 19 grips the work 130 and rotates or reverses the work 130 to change the posture of the work 130.

FIG. 4 is a flowchart showing a flow of processing in which the robot 10 grips the workpiece 130 and changes the posture by the workpiece posture changing device 5. When this processing is executed, first, the control device 50 analyzes the acquired CAD data of the workpiece 130 and extracts the CAD model 150 having a three-dimensional shape of the workpiece 130 (step S200).
Next, the control device 50 virtually changes the attitude of the CAD model 150 (step S202). Next, the control device 50 calculates the stability for each changed posture (step S204).
Next, the control device 50 projects the CAD model 150 to generate a template (step S206). Subsequently, the control device 50 stores the generated template in association with the posture stability (step S208).
Next, the control device 50 captures the workpiece 130 and acquires a captured image (step S215).

Next, the control device 50 obtains the posture of the work 130 by matching the captured image with the stored template (step S220).
Next, the control device 50 determines whether or not to warn the user from the characteristics of the workpiece 130 and the acquired stability of the posture (step S225).
Here, the control device 50 instructs processing according to the determination result (step S230). In other words, when it is determined to warn the user (Yes in step S230), the control device 50 warns the user with a message or light to watch (step S235), and ends the series of processes. On the other hand, if it is determined that no warning is required for the user (No in step S230), the control device 50 instructs the robot 10 to change the posture of the workpiece 130 (step S240) and ends the series of processes. To do.
Through the above-described processing, the stability of each of the postures that can be taken by the workpiece 130 is obtained, and the posture where the workpiece 130 is placed is recognized by performing pattern matching with the captured image of the workpiece 130, and the posture. Whether the posture of the workpiece 130 is changed by the robot 10 or whether the posture of the workpiece 130 is warned to the user is determined from the stability of the workpiece and the characteristics of the workpiece 130. Therefore, for example, in order to recognize the workpiece 130 that is unstablely placed on the belt conveyance device 135 so as to overlap with another workpiece 130, and to notify the user of a warning, the user can remove the unstable workpiece 130. Since it can be removed from the belt conveying device 135 or its posture can be corrected, the hand 19 of the robot 10 grips an unexpected part of the work 130, whereby the appearance of the work 130 is destroyed or the work 130 has. It can be avoided that the function is broken.

  5 ... Work posture change device, 10 ... Robot, 11 ... First axis, 12 ... Second axis, 13 ... Third axis, 14 ... Fourth axis, 15 ... Fifth axis, 16 ... Sixth axis, 17, 17A , 17B ... arm, 18 ... wrist, 19 ... hand, 20 ... digital camera, 25 ... robot base, 30 ... actuator, 50 ... control device, 80 ... computer, 82 ... display, 84 ... keyboard, 85 ... cable, 100 ... Template matching unit 105 ... Matching unit 106 ... Warning determination unit 108 ... Attitude correction instruction unit 109 ... Warning signal generation unit 110 ... Template acquisition unit 112 ... CAD data acquisition unit 114 ... Shape extraction unit 116 ... Projection unit 118 ... Template generation unit 120 ... Stability calculation unit 122 ... Template storage unit 124 ... Driver 126 ... Robot Control unit, 128 ... CAD database, 129 ... work information storage unit, 130 ... work, 132 ... indicator light control unit, 134 ... UI display control unit, 135 ... belt conveyor, 140 ... contact point calculation unit, 145 ... stable Sex determination unit, 150... CAD model, 160.

Claims (6)

An acquisition means for acquiring design data of the target;
Extraction means for extracting the shape of the target based on the acquired design data;
Projecting means for generating a plurality of projected images obtained by projecting the extracted shape onto a plane in a plurality of postures;
Stability when the target is placed for each of the plurality of postures based on a result of virtually changing the posture of the shape according to each of the plurality of postures and restoring the posture of the shape Calculating means for calculating
Template generating means for generating a template corresponding to each of the plurality of projection images;
Storage means for storing the generated template and the stability according to the template;
Photographing means for photographing the placed target and outputting a photographed image;
Matching means for selecting the template corresponding to the photographed image by comparing the photographed image with the template stored in the storage means;
Warning determination means for determining whether or not to warn about the posture on which the target is placed based on the stability stored in accordance with the selected template;
And a warning signal generation unit configured to generate a warning signal indicating a warning when it is determined to warn about the posture on which the target is placed. The template matching system according to claim 1,
The calculating means includes
A contact point calculation means for calculating a contact point of the shape that first contacts the plane when the shape is brought close to the plane in each of the plurality of postures;
Whether or not the posture of the shape returns to the original after virtually changing the posture of the shape placed on the plane with the contact point calculated in each of the plurality of postures as a fulcrum And a stability determining means for determining stability when the target is placed for each of the plurality of postures. The template matching system according to any one of claims 1 to 2,
The warning determination means determines whether or not to warn about the posture on which the target is placed based on the information related to the target in addition to the stability according to the template. Template matching system. The template matching system according to any one of claims 1 to 3,
The warning signal displays information indicating a warning on a display means. In the template matching system according to any one of claims 1 to 4,
And a posture change instruction means for instructing a posture change means capable of gripping the target to change the posture of the target when it is determined not to warn about the posture on which the target is placed. A template matching system. Acquiring the design data of the target;
Extracting the shape of the target based on the acquired design data;
Generating a plurality of projected images obtained by projecting the extracted shape onto a plane in a plurality of postures;
Stability when the target is placed for each of the plurality of postures based on a result of virtually changing the posture of the shape according to each of the plurality of postures and restoring the posture of the shape Calculating
Generating a template corresponding to each of the plurality of projection images;
Storing the generated template and the stability according to the template;
Photographing the placed target and outputting a photographed image; and
Selecting the template corresponding to the photographed image by comparing the stored template and the photographed image to determine the degree of coincidence;
Determining whether or not to warn about the posture on which the target is placed based on the stability stored in accordance with the selected template;
And a step of generating a warning signal indicating a warning when it is determined to warn about the posture on which the target is placed.

Images