TWI850888B - A measurement assistance system and method - Google Patents

Abstract

A measurement assistance system and method, comprising: a measurement platform with a working area for placing objects to be measured and measuring tools; a camera set on the measurement platform for obtaining measurement images; and a server module, Electrically connected to the camera, respectively execute the measuring tool recognition program, the measurement part recognition program, and the measurement posture recognition program according to the measurement image, and compare whether the appearance image of the measuring tool, the measurement part image, and the measurement posture image are correct; among them, the servo The instrument module has a processing unit, and when the image of the appearance of the measuring tool, the image of the measurement part, and the image of the measurement posture are all correct, a measurement result is generated according to the measurement data.

Description

Measurement assistance system and method

The present invention relates to a measurement auxiliary system and method, which is used for a measurement system for quality control and can identify and verify the accuracy of a part measurement location set by a user.

Currently, most visual recognition systems are used for product defect detection and product size detection. However, the entire production line visual recognition system is expensive, and it is inaccurate and difficult to measure parts above 2D. In addition, for small-scale production of high-precision products, it takes time to adjust and calibrate the recognition system.

Compared with the current quality control software, the current quality control measurement software can directly input the data of the measuring tool into the quality control software, and provide a fairly rich quality control chart to analyze the production line process capability. Quality control is based on data analysis based on the same data source. How to ensure the accuracy of the data when it is entered. Accuracy refers to whether the part measurement location, the type of measuring tool, etc. are correct, whether the hand is holding the tool, etc. The current measurement quality control software cannot make data judgments on the above-mentioned part location, tool type, and hand holding, etc. These errors will reduce the effectiveness of quality control.

The present invention provides a measurement assistance system and method, the purpose of which is to detect the data accuracy of quality control personnel when measuring parts based on a visual recognition assistance system.

The present invention provides a measurement auxiliary system, comprising: a measurement platform having a work area for placing a to-be-measured object and at least one measuring tool; at least one camera, disposed on the measurement platform, for obtaining a measurement image; and a server module, electrically connected to the cameras, for executing a measuring tool recognition program, a measurement part recognition program, and a measurement posture recognition program respectively according to the measurement image through a measuring tool appearance standard model, a measurement part standard model, and a measurement posture standard model, and obtaining a measuring tool appearance image corresponding to the measuring tools, a measurement part image of the to-be-measured object, and a measurement image of the to-be-measured object. The invention relates to a method for processing a measuring tool and a measuring posture image of a measurer to compare the measuring tool appearance image, the measuring part image, and the measuring posture image to see whether they are correct; wherein the server module has a processing unit, and when the measuring tool appearance image, the measuring part image, and the measuring posture image are all correct, a measurement result is generated according to a measurement data; wherein the measuring tool appearance standard model, the measuring part standard model, and the measuring posture standard model are trained through a pre-built deep learning neural network framework, and the deep learning neural network framework includes a Tensorflow object detection algorithm, a HU Moment algorithm, a TensorFlow CNN algorithm, and a MediaPipe hand algorithm.

In an embodiment of the present invention, the measurement auxiliary system further includes a display module, which is located in the working area and is used to display measurement information.

In one embodiment of the present invention, the measurement information includes a designated measuring tool, a measurement location, and the measurement result.

In one embodiment of the present invention, the measurement auxiliary system further includes an input module, which is used to obtain the measurement data.

In one embodiment of the present invention, the measurement auxiliary system further includes a judgment module, which is used to judge whether the measurement data meets a preset threshold range.

In an embodiment of the present invention, the aforementioned measuring tools include a wired measuring tool and/or a wireless measuring tool, and the wireless measuring tool has a corresponding measurement data receiving unit.

In one embodiment of the present invention, the gauge appearance standard model uses the TensorFlow algorithm as a deep learning neural network framework and is a neural network-like model trained by deep learning.

In one embodiment of the present invention, the above-mentioned measurement part standard model uses the HU Moment algorithm and the TensorFlow CNN algorithm as the deep learning neural network framework, and is a neural network model trained by deep learning.

In one embodiment of the present invention, the above-mentioned measured posture standard model uses the MediaPipe Hand algorithm as a method for hand recognition.

The present invention also includes a measurement assistance method, which is applicable to the above-mentioned measurement assistance system, wherein the measurement assistance method includes the following steps: displaying a measurement information through a display module; continuously acquiring a measurement image of a work area through at least one camera; executing a measurement tool recognition program through a server module based on the measurement image and a measurement tool appearance standard model to acquire a measurement tool appearance image and compare it with the measurement information; when the measurement tool appearance image is correct, executing a measurement part recognition program through a measurement part standard model based on the measurement image; A recognition procedure is performed to obtain a measurement part image and compare it with the measurement information; when the measurement part image is correct, a measurement posture recognition procedure is performed according to the measurement image through a measurement posture standard model to obtain a measurement posture image and compare it with the measurement information; when the measurement posture image is correct, a measurement data is obtained through an input module; a judgment module is used to judge whether the measurement data meets a preset threshold range; and when the measurement data meets the preset threshold range, a measurement result is generated according to the measurement data through a processing unit.

In an embodiment of the present invention, the aforementioned measuring tools include a wired measuring tool and/or a wireless measuring tool, and the wireless measuring tool has a corresponding measurement data receiving unit.

In one embodiment of the present invention, the aforementioned gauge appearance standard model uses the Tensorflow object detection algorithm as a deep learning neural network framework and a neural network-like model trained by deep learning.

In one embodiment of the present invention, the above-mentioned measurement part standard model uses the HU Moment algorithm and the TensorFlow CNN algorithm as the deep learning neural network framework, and is a neural network model trained by deep learning.

In one embodiment of the present invention, the above-mentioned measured posture standard model uses the MediaPipe hand algorithm as a method for hand recognition.

The effect of the present invention is that the visual recognition auxiliary system detects the accuracy of the quality control personnel when measuring parts. The auxiliary system is used in the field measurement system to visually recognize the part parts, measuring tools and postures measured by the field personnel, and detects their accuracy by machine visual recognition. Compared with the traditional measurement system, the user measures according to the set part part icon. If the measured parts are too similar, the measuring personnel will measure the wrong part, or the measuring personnel will not follow the standard measurement value and manually input erroneous data. In order to make up for this defect, this measurement auxiliary system integrates visual recognition with the measurement system, and uses visual recognition to judge its part parts, measuring tools and postures.

In order to make the above features and advantages of the present invention more obvious and easy to understand, the following embodiments are specifically cited and combined with the attached drawings. In addition to these detailed descriptions, the present invention can also be widely implemented in other embodiments. Any easy replacement, modification, and equivalent changes of the embodiments are included in the scope of the present invention and are subject to the scope of the patent application. In the description of the specification, many specific details are provided to enable readers to have a more complete understanding of the present invention; however, the present invention may still be implemented on the premise of omitting some or all of the specific details. In addition, well-known steps or components are not described in the details to avoid unnecessary limitations on the present invention. The same or similar components in the drawings will be represented by the same or similar symbols. It should be noted that the drawings are for illustration purposes only and do not represent the actual size or quantity of the components. Some details may not be fully drawn. In order to simplify the drawings, detailed descriptions are given below.

Please refer to FIG1. FIG1 is a block diagram of a measurement auxiliary system according to the present invention. A measurement auxiliary system of the present invention comprises: a measurement platform 11, having a work area 111 for placing a to-be-measured object and at least one measuring tool; at least one camera 12, disposed on the measurement platform 11, for acquiring a measurement image; and a server module 13, electrically connected to the cameras 12, and executing a measuring tool recognition program, a measurement part recognition program, and a measurement posture recognition program respectively according to the measurement image through a measuring tool appearance standard model, a measurement part standard model, and a measurement posture standard model, and acquiring a measuring tool appearance image corresponding to the measuring tools, a measurement image of the to-be-measured object, and a measurement image of the to-be-measured object. The server module 13 comprises a measuring part image and a measuring posture image of a measurer to compare whether the measuring tool appearance image, the measuring part image, and the measuring posture image are correct; wherein the server module 13 has a processing unit 131, and when the measuring tool appearance image, the measuring part image, and the measuring posture image are all correct, a measurement result is generated according to a measurement data; wherein the measuring tool appearance standard model, the measuring part standard model, and the measuring posture standard model are trained through a pre-built deep learning neural network framework, and the deep learning neural network framework includes a Tensorflow object detection algorithm, a HU Moment algorithm, a TensorFlow CNN algorithm, and a MediaPipe hand algorithm.

In this embodiment, the measurement auxiliary system further includes a display module 14, which is located in the working area 111 and is used to display measurement information.

The measurement information includes a designated measuring tool, a measurement location, and the measurement result.

In this embodiment, the measurement auxiliary system further includes a signal conversion device, which is connected to the server module 13 via the Internet, and the signal conversion device is connected to the display module 14 in a wired manner or a wireless manner.

The display module 14 may be, but is not limited to, a smart phone, a tablet computer, a desktop computer, or a notebook computer.

In this embodiment, the measurement auxiliary system further includes an input module 15, and the input module 15 is used to obtain the measurement data.

In this embodiment, the measurement auxiliary system further includes a judgment module 16, and the judgment module 16 is used to judge whether the measurement data meets a preset threshold range.

When the measured data exceeds the preset threshold range, a notification signal is sent out through an alarm unit.

In this embodiment, the measuring tools include a wired measuring tool and/or a wireless measuring tool, and the wireless measuring tool has a corresponding measurement data receiving unit.

In this embodiment, the gauge appearance standard model uses the Tensorflow object detection algorithm as a deep learning neural network framework and a neural network-like model trained by deep learning.

Among them, the Tensorflow object detection algorithm is an open source software library applied to machine learning, and TensorFlow provides a variety of artificial intelligence related models.

The gauge appearance standard model uses the Tensorflow object detection algorithm to identify objects, mainly measuring tools such as vernier calipers, separation cards, and height gauges, as well as the parts of the gauge that will contact the measured parts.

In this embodiment, the measurement site standard model uses the HU Moment algorithm and the TensorFlow CNN algorithm as a deep learning neural network framework, and is a neural network-like model trained by deep learning.

The HU Moment algorithm uses seven invariants of objects in the photo. The seven invariants are improved from standard geometric moments, so that the moments of the objects can remain unchanged after rotation, translation, scaling, mirroring, etc., which is more suitable for describing and comparing the similarity of objects than the original standard moments.

The seven invariants are as follows:

HU Moment remains the same after the object is rotated, translated, and scaled, and can be used for object shape recognition.

Among them, the measurement part standard model constructed by the HU Moment algorithm and the TensorFlow CNN algorithm can use part part images. Taking the vernier caliper as an example, different parts on the part will have different characteristics, and the part image is captured and identified based on the part touched by the main ruler.

The measurement part image refers to the image where the measuring tool contacts the measurement object.

In this embodiment, the measured pose standard model uses the MediaPipe hand algorithm as a method for hand recognition.

The measured posture image includes a hand posture and a finger angle.

The MediaPipe hand algorithm divides the hand into 21 feature points as shown in the figure below. The hand posture and finger angles can be derived from the feature points, and the hand posture is used as a basis for determining whether to grasp the measuring tool.

Please refer to FIG2 . FIG2 is a flow chart of a measurement assistance method according to the present invention. The measurement assistance method in FIG2 is applicable to a measurement assistance system, wherein the measurement assistance method comprises the following steps:

Step S210: Displaying measurement information via a display module;

Step S220: continuously acquiring a measurement image of a working area using at least one camera;

Step S230: executing a gauge recognition procedure through a server module according to the measurement image and a gauge appearance standard model to obtain a gauge appearance image and compare it with the measurement information;

Step S240: When the measuring tool appearance image is correct, a measurement part recognition procedure is performed based on the measurement image through a measurement part standard model to obtain a measurement part image, and compare it with the measurement information;

Step S250: When the measurement part image is correct, a measurement posture recognition procedure is performed based on the measurement image through a measurement posture standard model to obtain a measurement posture image, and compare it with the measurement information;

Step S260: When the measured posture image is correct, obtaining a measurement data through an input module;

Step S270: determining whether the measured data meets a preset threshold range through a determination module; and

Step S280: When the measurement data meets the preset threshold range, a measurement result is generated according to the measurement data through a processing unit.

In this embodiment, the measuring tools mentioned above include a wired measuring tool and/or a wireless measuring tool, and the wireless measuring tool has a corresponding measurement data receiving unit.

In this embodiment, the gauge appearance standard model uses the Tensorflow object detection algorithm as a deep learning neural network framework and a neural network-like model trained by deep learning.

In this embodiment, the above-mentioned measurement part standard model uses the HU Moment algorithm and the TensorFlow CNN algorithm as the deep learning neural network framework, and is a neural network-like model trained by deep learning.

In this embodiment, the above-mentioned measured posture standard model uses the MediaPipe hand algorithm as a method for hand recognition.

Please refer to FIG3. FIG3 is a schematic diagram of the judgment process of an embodiment of a measurement auxiliary method according to the present invention. In FIG3, the judgment process of the measurement auxiliary system firstly captures the image of the measurer on the measurement platform by the camera, and then identifies the measuring tool through the measuring tool recognition model based on the measuring tool appearance image to confirm whether the measuring tool is correct. If it is incorrect, the test fails, and if it passes, it enters the next stage; captures the measuring part image on the measuring tool, and then uses the Hu The moment calculated value is compared with the standard model of the measurement part constructed by CNN for identification. If it fails, the detection fails. If it passes, it enters the next stage; the measurement posture images of the hand holding and hand recognition are captured, and through the standard model of the measurement posture, it is identified whether the hand is holding the measuring tool. If it is incorrect, the detection is wrong, and if it is correct, the detection passes. A passed detection means that the user uses the correct measuring tool, measures the correct part on the part, and the hand confirms that the correct measuring tool is grasped.

In this embodiment, when all tests are passed, the measurement data is transmitted to the server module through the input module to generate the measurement results.

The judgment module is used to determine whether the measured data meets the preset threshold range.

In summary, the effect of the present invention is that the visual recognition auxiliary system detects the accuracy of the quality control personnel when measuring parts. The auxiliary system is used in the field measurement system to visually recognize the part position, measuring tool and posture measured by the field personnel, and detects its accuracy by machine visual recognition. Compared with the traditional measurement system, users can rely on Measuring according to the set part part diagram may cause the measuring personnel to measure the wrong part if the measured parts are too similar, or the measuring personnel may manually input incorrect data without following the specified measurement value. To make up for this defect, this measurement auxiliary system integrates visual recognition with the measurement system, and uses visual recognition to judge the part part, measuring tool and posture.

Although the present invention is disclosed as above with the aforementioned embodiments, they are not used to limit the present invention. Any person skilled in the art can make changes and modifications without departing from the spirit and scope of the present invention, and the equivalent substitutions are still within the scope of patent protection of the present invention.

11      Measurement platform

111         Work Area

12      Camera

13      Server Module

131 Processing unit

14      Display module

15 Input module

16      Judgment module

S210~S280         Steps

FIG1 is a block diagram of a measurement assistance system according to the present invention;

FIG2 is a flow chart of the steps of a measurement auxiliary method according to the present invention;

FIG3 is a schematic diagram of a determination process according to an embodiment of a measurement auxiliary method of the present invention.

11     Measurement platform 111   Work area 12     Camera 13     Server module 131   Processing unit 14     Display module 15     Input module 16     Judgment module

Claims (10)

A measurement auxiliary system includes: a measurement platform having a work area for placing a to-be-measured object and at least one measuring tool; at least one camera, disposed on the measurement platform, for obtaining a measurement image; and a server module, electrically connected to the cameras, for executing a measuring tool recognition program, a measurement part recognition program, and a measurement posture recognition program respectively according to the measurement image through a measuring tool appearance standard model, a measurement part standard model, and a measurement posture standard model, and obtaining a measuring tool appearance image corresponding to the measuring tools, a measurement part image of the to-be-measured object, and a measurement posture image of a measurer, for comparing the appearance image of the measuring tool, the measurement part image, and the measurement posture image to see if they are correct; wherein the server module has a processing unit, and when the appearance image of the measuring tool, the measurement part image, and the measurement posture image are all correct, a measurement result is generated according to a measurement data; wherein the appearance standard model of the measuring tool, the measurement part standard model, and the measurement posture standard model are trained by a pre-built deep learning neural network framework, and the deep learning neural network framework includes Tensorflow The object detection algorithm is used to identify objects and the measuring tools. The HU Moment algorithm is used to describe and compare the similarity of objects and compare the appearance image of the measuring tool. The TensorFlow CNN algorithm and the MediaPipe hand algorithm are used to divide the measurement posture image into multiple feature points. The hand posture and finger angle are derived from these feature points, and the hand posture is used to determine whether to grasp the measuring tool. The measurement auxiliary system as described in item 1 of the patent application scope, wherein the measurement auxiliary system further includes a display module, the display module is located in the working area, and is used to display measurement information. A measurement assistance system as described in item 2 of the patent application, wherein the measurement information includes a specified measuring tool, a measurement location, and the measurement result. The measurement auxiliary system as described in item 1 of the patent application scope, wherein the measurement auxiliary system further includes an input module, and the input module is used to obtain the measurement data. As described in item 1 of the patent application scope, the measurement auxiliary system further includes a judgment module, which is used to judge whether the measurement data meets a preset threshold range. As described in item 1 of the patent application scope, the measuring tools include a wired measuring tool and/or a wireless measuring tool, and the wireless measuring tool has a corresponding measurement data receiving unit. As described in item 1 of the patent application scope, the measurement auxiliary system, wherein the gauge appearance standard model uses the Tensorflow object detection algorithm as a deep learning neural network framework and is a neural network-like model trained by deep learning. As described in the first item of the patent application scope, the measurement auxiliary system, wherein the measurement part standard model uses the HU Moment algorithm and the TensorFlow CNN algorithm as the deep learning neural network framework, and is a neural network-like model trained by deep learning. The measurement assistance system as described in item 1 of the patent application scope, wherein the measurement posture standard model uses the MediaPipe hand algorithm as a method for hand recognition. A measurement assistance method is applicable to the measurement assistance system of any one of claim 1 to claim 9, wherein the measurement assistance method comprises the following steps: displaying measurement information through a display module; continuously acquiring a measurement image of a work area through at least one camera; executing a measurement tool recognition program through a measurement tool appearance standard model according to the measurement image through a server module to acquire a measurement tool appearance image, and compare it with the measurement information; when the measurement tool appearance image is correct, executing a measurement part recognition program through a measurement part standard model according to the measurement image to acquire a measurement part image, and compare it with the measurement information; when the measurement part image is correct, executing a measurement part recognition program through a measurement part standard model according to the measurement image to acquire a measurement part image, and compare it with the measurement information; The measurement image executes a measurement posture recognition procedure through a measurement posture standard model to obtain a measurement posture image and compare it with the measurement information; when the measurement posture image is correct, a measurement data is obtained through an input module; a judgment module is used to judge whether the measurement data meets a preset threshold range; and when the measurement data meets the preset threshold range, a measurement result is generated according to the measurement data through a processing unit; wherein the gauge appearance standard model, the measurement part standard model, and the measurement posture standard model are trained through a pre-built deep learning neural network framework, and the deep learning neural network framework includes Tensorflow The object detection algorithm is used to identify objects and the measuring tools. The HU Moment algorithm is used to describe and compare the similarity of objects and compare the appearance image of the measuring tool. The TensorFlow CNN algorithm and the MediaPipe hand algorithm are used to divide the measurement posture image into multiple feature points, and the hand posture and finger angle are obtained from these feature points. The hand posture is used as the basis for grasping the measuring tool.