JP6829271B2 - Measurement operation parameter adjustment device, machine learning device and system - Google Patents

Description

The present invention relates to a measurement operation parameter adjusting device, a machine learning device and a system.

Conventionally, as illustrated in FIG. 11, a contact type touch sensor or the like or a non-contact type visual sensor or the like is attached to a mechanical mechanical part, and the sensor is moved by moving the mechanical part for measurement. The position of the object is being measured. In the example shown in FIG. 11, when the contact type sensor is used, the position of the object to be measured is measured based on the coordinate value of the mechanical part when the contact type sensor detects that the work is in contact with the work, and the contact is not made. When a sensor is used, the position of the object to be measured is measured based on the coordinate value of the mechanical part when the distance of the object facing the non-contact sensor changes, that is, when the non-contact sensor approaches the end of the object to be measured. Can be measured.

When detecting the position of the object to be measured by the sensor attached to the mechanical mechanism, it is necessary to determine in advance the position of the mechanism at the start of the measurement operation and the moving speed of the mechanism during the measurement operation. There is. As a conventional technique for determining such a position and speed, for example, in Patent Document 1, a sensor is brought into contact with a measurement object before measurement is performed, and a mechanism unit at the start of measurement operation is based on the position at that time. A technique for determining a position or the like is disclosed.

Japanese Unexamined Patent Publication No. 2010-217182

Although the technique disclosed in Patent Document 1 is effective when there is no variation in the size and installation position of the object to be measured, in the actual field, the position where the contact sensor contacts and the non-contact sensor change the distance. The detection position varies even if the measurement target is of the same type. For example, when measuring the position of a measurement object having a hole at the bottom and fitting the hole into a protrusion provided on a table, ideally, the hole is always installed at the same position. However, in reality, a play is provided between the hole and the protrusion, and the installation position of the object to be measured is slightly displaced. In addition, the shape of the object to be measured may vary, or the position of the object to be measured of different types may be measured by the same measuring mechanism. In order to deal with such a situation, in the technique disclosed in Patent Document 1, there is no choice but to uniformly set the measurement operation start position with a margin assuming the case where the variation is the largest, depending on the situation. It is difficult to shorten the tact time.

In addition, the time it takes for the contact sensor to come into contact with the object to be measured and detect the digital signal, and the time it takes for the non-contact sensor to reach the end of the object to be measured until the digital signal is detected are determined by the sensor. Even if the type is the same, there are variations due to individual differences, and if the movement speed of the sensor is not set in consideration of this variation, the (contact type) sensor will hit the object to be measured and the sensor, mechanism, etc. There may be a problem that the object to be measured is damaged or a problem that the installation position of the object to be measured is displaced. Furthermore, if the speed at which the sensor is moved in the measurement operation is not a constant speed, the position of the object to be measured will vary, so the speed at which the sensor is moved in the measurement operation is set to a constant slow speed when the whole is taken into consideration. However, there is another problem that the tact time is delayed.

Therefore, a measurement operation parameter adjusting device, a machine learning device, and a system that enable efficient measurement of the installation position of the measurement object even if there are variations in the installation position, size, and type of the measurement object are provided. It is desired.

One aspect of the present invention is a measurement operation parameter adjusting device that adjusts the measurement operation parameters of the measurement operation executed by the measurement device that measures the installation position of the measurement object, and is a measurement indicating the measurement operation parameters of the measurement operation. An operation parameter data and measurement time data indicating the time required for the measurement operation are observed as state variables representing the current state of the environment, and a learning model modeling the adjustment of the measurement operation parameters based on the state variables is obtained. A machine learning device for learning or making a decision is provided , and the machine learning device obtains measurement operation parameter data indicating the measurement operation parameters of the measurement operation and measurement time data indicating the time required for the measurement operation in an environment. The state observation unit that observes as a state variable representing the current state, and the measurement operation judgment data that determines the suitability of the measurement operation executed based on the measurement operation parameters of the measurement operation are determined to indicate the suitability judgment result of the measurement operation. Using the determination data acquisition unit acquired as data, the state variable, and the determination data, the learning model learned by associating the time required for the measurement operation with the adjustment of the measurement operation parameter of the measurement operation is generated. a learning unit for, Ru provided with a measurement operation parameter adjustment device.

In another aspect of the present invention, measurement operation parameter data indicating the measurement operation parameters of the measurement operation executed by the measurement device for measuring the installation position of the measurement object, and measurement time data indicating the time required for the measurement operation are obtained. A machine learning device that observes as a state variable representing the current state of the environment, and based on the state variable, learns or makes a decision using a learning model that models the adjustment of the measurement operation parameter. Measurement of operation The measurement operation parameter data indicating the operation parameter and the measurement time data indicating the time required for the measurement operation are used as the state observation unit for observing the current state of the environment as a state variable and the measurement operation parameter of the measurement operation. Using the determination data acquisition unit that acquires the measurement operation determination data for determining the suitability of the measurement operation executed based on the determination data as the determination data indicating the suitability determination result of the measurement operation, and the state variable and the determination data, It is a machine learning device including a learning unit for generating the learning model learned by associating the time required for the measurement operation with the adjustment of the measurement operation parameters of the measurement operation.

Another aspect of the present invention is a system in which a plurality of devices are connected to each other via a network, and the plurality of devices include at least the first measurement operation parameter adjusting device according to claim 1. It is a system.

In another aspect of the present invention, the measurement operation parameter data indicating the measurement operation parameter of the measurement operation for measuring the installation position of the measurement object and the measurement time data indicating the time required for the measurement operation represent the current state of the environment. The step of observing as a state variable and learning or making a decision using a learning model that models the adjustment of the measurement operation parameter based on the state variable is executed by a measuring device that measures the installation position of the measurement object. It is a method of learning or making a decision on adjustment of measurement motion parameters to be performed, and a step of learning or making a decision using the learning model is a measurement motion parameter data indicating the measurement motion parameter of the measurement motion and the measurement. The step of observing the measurement time data indicating the time required for the operation as a state variable representing the current state of the environment, and the measurement operation judgment data for determining the suitability of the measurement operation executed based on the measurement operation parameters of the measurement operation. , The step of acquiring as the judgment data indicating the suitability judgment result of the measurement operation, and the time required for the measurement operation and the adjustment of the measurement operation parameter of the measurement operation are associated with each other by using the state variable and the judgment data. It is a method of learning or making a decision on the adjustment of measurement motion parameters to be executed .

According to the present invention, even if there are variations in the installation position, size, and product type of the measurement object, the installation position of the measurement object can be efficiently measured by learning the degree of variation with a machine learning device. Is possible.

It is a schematic hardware block diagram of the measurement operation parameter adjustment apparatus by one Embodiment. It is a schematic functional block diagram of the measurement operation parameter adjustment apparatus by one Embodiment. It is a schematic functional block diagram which shows one form of the measurement operation parameter adjustment apparatus. It is a schematic flowchart which shows one form of the machine learning method. It is a figure explaining a neuron. It is a figure explaining a neural network. It is a figure which shows the example of the system of the three-layer structure including a cloud server, a fog server, and an edge computer. It is a schematic functional block diagram which shows one form of the system which incorporated the measurement operation parameter adjustment device. It is a schematic functional block diagram which shows other form of the system which incorporated the measurement operation parameter adjustment device. It is a schematic hardware configuration diagram of the computer shown in FIG. It is a schematic functional block diagram which shows other form of the system which incorporated the measurement operation parameter adjustment device. It is a figure explaining the measurement operation of the installation position of the measurement object by a measuring device.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a schematic hardware configuration diagram showing a main part of a measurement operation parameter adjusting device according to an embodiment. The measurement operation parameter adjusting device 1 can be mounted as, for example, a control device for controlling a manufacturing machine such as a robot, or a personal computer attached to the manufacturing machine or the like. Further, the measurement operation parameter adjusting device 1 can be implemented as a computer such as a cell computer, an edge computer, a fog computer, a host computer, or a cloud server connected to a manufacturing machine or the like via a wired / wireless network. In this embodiment, an example is shown in which the measurement operation parameter adjusting device 1 is mounted as a personal computer attached to a manufacturing machine or the like.

The CPU 11 included in the measurement operation parameter adjusting device 1 according to the present embodiment is a processor that controls the measurement operation parameter adjusting device 1 as a whole. The CPU 11 reads the system program stored in the ROM 12 via the bus 20 and controls the entire measurement operation parameter adjusting device 1 according to the system program. The RAM 13 temporarily stores temporary calculation data, display data to be displayed on the display device 70 such as a display, and various data input by the operator via the input device 71 such as a keyboard, mouse, and touch panel.

The non-volatile memory 14 is configured as a memory in which the storage state is maintained even when the power of the measurement operation parameter adjusting device 1 is turned off, for example, by backing up with a battery (not shown). In the non-volatile memory 14, various data (for example, the measuring device 2) acquired from the measuring device 2 such as a program input via the input device 71, each part of the measuring operation parameter adjusting device 1 and a robot holding the sensor The position of the measuring device 2 in the coordinate system to be measured, the position of the sensor provided in the measuring device 2 at the start of the measurement operation, the moving speed of the sensor provided in the measuring device 2 during the measurement operation, and the sensor during the measurement operation are the objects to be measured. The time from the measurement operation meeting instruction until the detection is detected, the type of the measurement object, the state information of the sensor and the measurement object input by the operator, etc.) are stored. The program and various data stored in the non-volatile memory 14 may be expanded in the RAM 13 at the time of execution / use. Further, various system programs (including a system program for controlling interaction with the machine learning device 100, which will be described later) such as a known analysis program are written in advance in the ROM 12.

The interface 21 is an interface for connecting the measurement operation parameter adjusting device 1 and the machine learning device 100. The machine learning device 100 stores a processor 101 that controls the entire machine learning device 100, a ROM 102 that stores a system program and the like, a RAM 103 that temporarily stores each process related to machine learning, a learning model, and the like. The non-volatile memory 104 used for the above is provided. The machine learning device 100 has each information that can be acquired by the measurement operation parameter adjusting device 1 via the interface 21 (for example, the position of the measuring device 2 in the coordinate system measured by the measuring device 2 and the sensor included in the measuring device 2). The position at the start of the measurement operation, the moving speed of the sensor provided in the measuring device 2 during the measurement operation, the time from the measurement operation meeting instruction until the sensor during the measurement operation detects the measurement object, the type of the measurement object, and the work. It is possible to observe the sensor input by the person, the state information of the object to be measured, etc.). Further, the measurement operation parameter adjusting device 1 receives the adjustment command of the measurement operation parameter output from the machine learning device 100, and sets the measurement operation parameter by the measuring device 2.

FIG. 2 is a schematic functional block diagram of the measurement operation parameter adjusting device 1 and the machine learning device 100 according to the embodiment. In each functional block shown in FIG. 2, the CPU 11 included in the measurement operation parameter adjusting device 1 shown in FIG. 1 and the processor 101 of the machine learning device 100 execute their respective system programs, and the measurement operation parameter adjusting device 1 And it is realized by controlling the operation of each part of the machine learning device 100.

The measurement operation parameter adjusting device 1 of the present embodiment includes a measurement operation parameter setting unit 34 that controls the measuring device 2 based on the measurement operation parameter adjustment command output from the machine learning device 100. The measurement operation parameter setting unit 34 is a functional means for adjusting the measurement operation parameters used in the measurement operation by the measurement device 2. The measurement operation parameter setting unit 34 can set at least the measurement operation start position and the measurement operation speed when the measurement device 2 measures the position of the measurement object.

On the other hand, the machine learning device 100 included in the measurement operation parameter adjusting device 1 is software (learning algorithm, etc.) and hardware for self-learning the adjustment of the measurement operation parameter of the measurement operation with respect to the time required for the measurement operation by so-called machine learning. (Processor 101 and the like) are included. What the machine learning device 100 included in the measurement operation parameter adjusting device 1 learns corresponds to a model structure showing a correlation between the time required for the measurement operation and the adjustment of the measurement operation parameter of the measurement operation.

As shown by the functional block in FIG. 2, the machine learning device 100 included in the measurement operation parameter adjusting device 1 has measurement operation parameter data S1 indicating the measurement operation parameter of the measurement operation and measurement time data S2 indicating the time required for the measurement operation. The state observation unit 106 for observing as the state variable S representing the current state of the environment including the above, and the measurement operation determination data D1 for determining the quality of the measurement operation executed based on the measurement operation parameters of the adjusted measurement operation. A determination data acquisition unit 108 for acquiring the included determination data D, and a learning unit 110 for learning by associating the time required for the measurement operation with the adjustment of the measurement operation parameters of the measurement operation using the state variable S and the determination data D. To be equipped.

Of the state variables S observed by the state observation unit 106, the measurement operation parameter data S1 can be acquired as the measurement operation parameters of the measurement operation performed by the measurement device 2. Examples of the measurement operation parameters of the measurement operation include the position of the sensor at the start of the measurement operation by the measurement device 2, the moving speed of the sensor in the measurement operation by the measurement device 2, and the like. These measurement operation parameters are set in the measurement operation parameter adjustment device 1, the current parameters set in the measurement device 2, the parameters obtained by analyzing the program for operating the measurement device 2, the non-volatile memory 14, etc. It can be acquired from the measurement operation parameters and the like set in the measuring device 2 before one learning cycle stored in the memory of.

The measurement operation parameter data S1 is a measurement operation parameter of the measurement operation adjusted in the learning cycle with respect to the time required for the measurement operation in the previous learning cycle by the machine learning device 100 based on the learning result of the learning unit 110. It can be used as it is. When such a method is adopted, the machine learning device 100 temporarily stores the measurement operation parameters of the measurement operation in the RAM 103 for each learning cycle, and the state observation unit 106 temporarily stores the measurement operation parameters of the measurement operation in the learning cycle immediately before the RAM 103. The measurement operation parameter of the measurement operation in the above may be acquired as the measurement operation parameter data S1 of the current learning cycle.

Of the state variables S observed by the state observation unit 106, the measurement time data S2 is from the start of the measurement operation executed by the measurement device 2 recorded in the non-volatile memory 14 until the sensor detects the object to be measured. You can get the time taken as.

The determination data acquisition unit 108 can use, as the measurement operation determination data D1, the determination result of suitability of the measurement operation when the measurement operation is performed based on the measurement operation parameters of the adjusted measurement operation. The measurement operation determination data D1 used by the determination data acquisition unit 108 includes, for example, that the moving distance of the sensor during the measurement operation is smaller than a predetermined threshold value when the measurement operation is performed with the adjusted measurement operation parameters. (Appropriate) or large (No), whether the object to be measured or the mechanical part or sensor of the measuring device 2 is damaged during the measurement operation (No), or whether the object to be measured has moved during the measurement operation (No) You can use something like.

The determination data acquisition unit 108 has an indispensable configuration at the stage of learning by the learning unit 110, but the learning that associates the time required for the measurement operation by the learning unit 110 with the adjustment of the measurement operation parameters of the measurement operation is completed. Later, it is not always an essential configuration. For example, when the machine learning device 100 for which learning has been completed is shipped to the customer, the determination data acquisition unit 108 may be removed before shipping.

The state variable S simultaneously input to the learning unit 110 is based on the data one learning cycle before the determination data D was acquired, when considered in the learning cycle by the learning unit 110. In this way, while the machine learning device 100 included in the measurement operation parameter adjusting device 1 advances learning, in the environment, the measurement time data S2 is acquired, and based on the measurement operation parameter data S1 adjusted based on each acquired data. The measurement operation is executed by the measuring device 2 and the determination data D is acquired repeatedly.

The learning unit 110 learns to adjust the measurement operation parameters of the measurement operation with respect to the time required for the measurement operation according to an arbitrary learning algorithm collectively called machine learning. The learning unit 110 can repeatedly execute learning based on the data set including the above-mentioned state variable S and the determination data D. During the repetition of the learning cycle of the measurement motion parameters of the measurement motion with respect to the time required for the measurement motion, the state variable S is the time required for the measurement motion one learning cycle before and the measurement adjusted before one learning cycle as described above. The determination data D obtained from the measurement operation parameters of the operation is the suitability determination result of the measurement operation performed based on the measurement operation parameters of the adjusted measurement operation.

By repeating such a learning cycle, the learning unit 110 can identify a feature that implies a correlation between the time required for the measurement operation and the adjustment of the measurement operation parameter of the measurement operation. At the start of the learning algorithm, the correlation between the time required for the measurement operation and the adjustment of the measurement operation parameters of the measurement operation is practically unknown, but the learning unit 110 gradually identifies the features and correlates as the learning progresses. To interpret. When the correlation between the time required for the measurement operation and the adjustment of the measurement operation parameters of the measurement operation is interpreted to a certain reliable level, the learning result repeatedly output by the learning unit 110 depends on the current state (that is, the measurement operation). It can be used to make action selections (ie, decisions) on how to adjust the measurement motion parameters of the measurement motion with respect to time). That is, as the learning algorithm progresses, the learning unit 110 gradually brings the correlation with the action of how to adjust the measurement motion parameter of the measurement motion with respect to the time required for the measurement motion closer to the optimum solution. Can be done.

The decision-making unit 122 adjusts the measurement operation parameter of the measurement operation based on the result learned by the learning unit 110, and outputs the measurement operation parameter of the adjusted measurement operation to the measurement operation parameter setting unit 34. When the time required for the measurement operation is input to the machine learning device 100 at the stage when the learning by the learning unit 110 becomes available for adjusting the measurement operation parameter, the decision unit 122 determines the measurement operation parameter of the measurement operation. (At least one of the position of the sensor at the start of the measurement operation and the moving speed of the sensor, etc.) is output. The decision-making unit 122 adjusts the measurement operation parameters of the appropriate measurement operation based on the state variable S and the result learned by the learning unit 110.

As described above, in the machine learning device 100 included in the measurement operation parameter adjusting device 1, the learning unit 110 uses the state variable S observed by the state observation unit 106 and the determination data D acquired by the determination data acquisition unit 108. According to the machine learning algorithm, the adjustment of the measurement operation parameter of the measurement operation with respect to the time required for the measurement operation is learned. The state variable S is composed of data such as measurement operation parameter data S1 and measurement time data S2, and the determination data D is uniquely obtained from the information acquired during the measurement operation by the measurement device 2. Therefore, according to the machine learning device 100 included in the measurement operation parameter adjusting device 1, by using the learning result of the learning unit 110, the measurement operation parameter of the measurement operation is automatically adjusted according to the time required for the measurement operation. And it is possible to do it accurately.

Then, if the measurement operation parameters of the measurement operation can be automatically adjusted, the appropriate value of the measurement operation parameters of the measurement operation can be quickly obtained only by grasping the time required for the measurement operation (measurement time data S2). Can be adjusted. Therefore, the measurement operation parameters of the measurement operation can be efficiently adjusted.

As a modification of the machine learning device 100 included in the measurement operation parameter adjusting device 1, the state observing unit 106, in addition to the measurement operation parameter data S1 and the measurement time data S2, has the product type data S3 showing the product type information of the measurement object. May be observed as the state variable S. As the product type data S3, an identification number or the like assigned so that the type of the object to be measured is uniquely determined can be used. By using the product type data S3 as the state variable S, it becomes possible to learn the position and moving speed of the sensor at the start of the measurement operation according to the product type of the measurement target object and reflect it in the adjustment of the measurement operation parameters. ..

As another modification of the machine learning device 100 included in the measurement operation parameter adjusting device 1, the state observation unit 106 adds the measurement operation parameter data S1 and the measurement time data S2 to the time zone and season when the measurement operation is executed. , The measurement environment data S4 indicating the measurement environment such as temperature and humidity may be observed as the state variable S. Depending on the object to be measured, there are objects (processed foods such as bread and rice cakes, fruits, vegetables, etc.) whose size and strength vary depending on temperature, humidity, time, etc., and measurement to observe such objects In the operation, by observing and using the measurement environment data S4 as a state variable, more appropriate measurement operation parameters can be adjusted.

In the machine learning device 100 having the above configuration, the learning algorithm executed by the learning unit 110 is not particularly limited, and a learning algorithm known as machine learning can be adopted. FIG. 3 is a form of the measurement operation parameter adjusting device 1 shown in FIG. 2, and shows a configuration including a learning unit 110 that executes reinforcement learning as an example of a learning algorithm. Reinforcement learning is a trial-and-error cycle of observing the current state (that is, input) of the environment in which the learning target exists, executing a predetermined action (that is, output) in the current state, and giving some reward to that action. It is a method of repeatedly learning a measure that maximizes the total reward (measurement operation parameter of measurement operation in the machine learning device of the present application) as an optimum solution.

In the machine learning device 100 included in the measurement operation parameter adjusting device 1 shown in FIG. 3, the learning unit 110 adjusts the measurement operation parameters of the measurement operation based on the state variable S, and sets the adjusted measurement operation parameters of the measurement operation. Using the reward calculation unit 112 for obtaining the reward R related to the suitability judgment result (corresponding to the judgment data D used in the next learning cycle in which the state variable S is acquired) of the measurement operation by the measuring device 2 based on the measurement device 2, and the reward R. , A value function update unit 114 that updates a function Q representing the value of the measurement operation parameter of the measurement operation is provided. The learning unit 110 learns the adjustment of the measurement operation parameter of the measurement operation with respect to the time required for the measurement operation by the value function update unit 114 repeating the update of the function Q.

An example of the reinforcement learning algorithm executed by the learning unit 110 will be described. The algorithm according to this example is known as Q-learning, and acts in the state s with the state s of the action subject and the action a that the action subject can select in the state s as independent variables. This is a method of learning a function Q (s, a) that represents the value of an action when a is selected. The optimum solution is to select the action a having the highest value function Q in the state s. By starting Q-learning in a state where the correlation between the state s and the action a is unknown and repeating trial and error to select various actions a in an arbitrary state s, the value function Q is repeatedly updated and optimized. Get closer to the solution. Here, when the environment (that is, the state s) changes as a result of selecting the action a in the state s, a reward (that is, weighting of the action a) r corresponding to the change is obtained, and a higher reward is obtained. By inducing learning to select the action a in which r is obtained, the value function Q can be approached to the optimum solution in a relatively short time.

The update formula of the value function Q can be generally expressed as the following formula 1. In equation (1), s _t and a _t is a state and behavior at each time t, the state by action a _t is changed to s _{t + 1.} r t _{+ 1} is a reward obtained by the state changes from s _t in s _{t + 1.} The term of maxQ means the Q when the action a which becomes the maximum value Q at the time t + 1 (which is considered at the time t) is performed. α and γ are learning coefficients and discount rates, respectively, and are arbitrarily set with 0 <α ≦ 1 and 0 <γ ≦ 1.

When the learning unit 110 executes Q learning, the state variable S observed by the state observation unit 106 and the judgment data D acquired by the judgment data acquisition unit 108 correspond to the update type state s and correspond to the current state (that is, measurement). The action of how to adjust the measurement motion parameter of the measurement motion with respect to the time required for the motion) corresponds to the update-type action a, and the reward R required by the reward calculation unit 112 corresponds to the update-type reward r. To do. Therefore, the value function update unit 114 repeatedly updates the function Q representing the value of the measurement operation parameter of the measurement operation with respect to the current state by Q learning using the reward R.

For the reward R obtained by the reward calculation unit 112, for example, the suitability determination result of the measurement operation performed based on the measurement operation parameter of the adjusted measurement operation, which is performed after adjusting the measurement operation parameter of the measurement operation, is “appropriate”. When it is determined that (for example, when the time required for the measurement operation is less than or equal to a predetermined threshold value), a positive (plus) reward R is set, and the adjustment is performed after adjusting the measurement operation parameters of the measurement operation. When the suitability judgment result of the measurement operation performed based on the measurement operation parameter of the measured operation is determined to be "No" (for example, when the time required for the measurement operation exceeds a predetermined threshold value, the sensor When the object to be measured collides with the object to be measured, or when the object to be measured has moved, etc.), the reward R can be negative (minus). The absolute values of the positive and negative rewards R may be the same or different from each other. Further, as a judgment condition, a plurality of values included in the judgment data D may be combined for judgment.

Further, the suitability determination result of the measurement operation based on the adjusted measurement operation parameter can be set not only in two ways of "appropriate" and "no" but also in a plurality of stages. As an example, when the threshold value of the time required for the measurement operation is T _max , when the time T required for the measurement operation is 0 ≦ T <T _max / 5, the reward R = 5 is given, and T _max / 5 ≦ T < When T _max / 2, reward R = 3, give reward R = 1 when T _max / 2 ≤ T <T _max , and when T _max ≤ T, reward R = -3 (minus reward). It can be configured to give.

Further, the threshold value used for the determination may be set relatively large in the initial stage of learning, and the threshold value used for the determination may be reduced as the learning progresses.

The value function update unit 114 can have an action value table in which the state variable S, the determination data D, and the reward R are arranged in association with the action value (for example, a numerical value) represented by the function Q. In this case, the act of the value function update unit 114 updating the function Q is synonymous with the act of the value function update unit 114 updating the action value table. Since the correlation between the current state of the environment and the adjustment of the measurement operation parameter of the measurement operation is unknown at the start of Q learning, various state variables S, judgment data D, and reward R are absent in the action value table. It is prepared in a form associated with the action value value (function Q) determined by the act. If the determination data D is known, the reward calculation unit 112 can immediately calculate the reward R corresponding to the determination data D, and the calculated value R is written in the action value table.

When Q-learning is advanced using the reward R according to the suitability determination result of the operation of the measuring device 2, the learning is guided in the direction of selecting the action that can obtain the higher reward R, and the selected action is executed in the current state. The action value value (function Q) for the action performed in the current state is rewritten and the action value table is updated according to the state of the environment that changes as a result (that is, the state variable S and the determination data D). By repeating this update, the value of the action value (function Q) displayed in the action value table is measured by proper action (in the case of the present invention, the object to be measured, the mechanical part of the measuring device 2 or the sensor is broken, or the sensor is broken. The larger the value, the larger the value (action to adjust the measurement operation parameters of the measurement operation, such as moving the sensor position at the start of the measurement operation closer to the measurement object or increasing the movement speed of the sensor within the range where the object does not move). It is rewritten so that In this way, the correlation between the current state of the unknown environment (time required for the measurement operation) and the behavior for it (adjustment of the measurement operation parameters of the measurement operation) is gradually clarified. In other words, by updating the action value table, the relationship between the time required for the measurement operation and the adjustment of the measurement operation parameters of the measurement operation is gradually brought closer to the optimum solution.

With reference to FIG. 4, the above-mentioned Q-learning flow (that is, one form of the machine learning method) executed by the learning unit 110 will be further described. First, in step SA01, the value function update unit 114 adjusts the measurement operation parameter of the measurement operation as an action to be performed in the current state indicated by the state variable S observed by the state observation unit 106 while referring to the action value table at that time. Randomly select actions. Next, the value function update unit 114 takes in the state variable S of the current state observed by the state observation unit 106 in step SA02, and the determination data of the current state acquired by the determination data acquisition unit 108 in step SA03. Take in D. Next, the value function update unit 114 determines in step SA04 whether or not the measurement operation based on the measurement operation parameters of the adjusted measurement operation is appropriate based on the determination data D, and if it is appropriate, in step SA05. , The positive reward R obtained by the reward calculation unit 112 is applied to the update formula of the function Q, and then in step SA06, the state variable S in the current state, the determination data D, the reward R, and the value of the action value (the updated function). Update the action value table using Q) and. When it is determined in step SA04 that the measurement operation based on the measurement operation parameter of the adjusted measurement operation is not appropriate, the negative reward R obtained by the reward calculation unit 112 is applied to the update formula of the function Q in step SA07. Next, in step SA06, the action value table is updated using the state variable S in the current state, the determination data D, the reward R, and the action value value (updated function Q). The learning unit 110 iteratively updates the action value table by repeating steps SA01 to SA07, and advances learning for adjusting the measurement motion parameters of the measurement motion. The process of obtaining the reward R and the process of updating the value function from step SA04 to step SA07 are executed for each data included in the determination data D.

For example, a neural network can be applied when advancing the reinforcement learning described above. FIG. 5A schematically shows a neuron model. FIG. 5B schematically shows a model of a three-layer neural network constructed by combining the neurons shown in FIG. 5A. The neural network can be configured by, for example, an arithmetic unit or a storage device that imitates a neuron model.

The neuron shown in FIG. 5A outputs the result y for a plurality of inputs x (here, as an example, inputs x ₁ to inputs x ₃ ). Each input x ₁ ~x _3, the weight w corresponding to the input x (w ₁ ~w ₃₎ is multiplied. As a result, the neuron outputs the output y expressed by the following equation (2). In the equation 2, the input x, the output y, and the weight w are all vectors. Further, θ is a bias and f _k is an activation function.

In the three-layer neural network shown in FIG. 5B, a plurality of inputs x (here, as an example, inputs x1 to input x3) are input from the left side, and a result y (here, as an example, result y1 to result y3) is input from the right side. It is output. In the illustrated example, each of the inputs x1, x2, x3 is multiplied by the corresponding weight (collectively represented by W1), and the individual inputs x1, x2, x3 are all three neurons N11, N12, N13. It has been entered.

In FIG. 5B, the outputs of neurons N11 to N13 are collectively represented by z1. z1 can be regarded as a feature vector obtained by extracting the feature amount of the input vector. In the illustrated example, each of the feature vectors z1 is multiplied by a corresponding weight (collectively represented by W2), and each of the individual feature vectors z1 is input to the two neurons N21 and N22. The feature vector z1 represents a feature between the weights W1 and W2.

In FIG. 5B, the outputs of the neurons N21 to N22 are collectively represented by z2. z2 can be regarded as a feature vector extracted from the feature amount of the feature vector z1. In the illustrated example, each of the feature vectors z2 is multiplied by a corresponding weight (collectively represented by W3), and each of the individual feature vectors z2 is input to the three neurons N31, N32, and N33. The feature vector z2 represents a feature between the weights W2 and W3. Finally, the neurons N31 to N33 output the results y1 to y3, respectively.
It is also possible to use a so-called deep learning method that uses a neural network consisting of three or more layers.

In the machine learning device 100 included in the measurement operation parameter adjusting device 1, the neural network is used as a value function in Q-learning, the state variable S and the action a are input x, and the learning unit 110 has a multi-layer structure according to the above-mentioned neural network. By performing the calculation, the value (result y) of the action in the state can be output. The operation mode of the neural network includes a learning mode and a value prediction mode. For example, the weight w is learned using the learning data set in the learning mode, and the value of the action is used in the value prediction mode using the learned weight w. You can make a judgment. In the value prediction mode, detection, classification, inference, and the like can also be performed.

The configuration of the measurement operation parameter adjusting device 1 described above can be described as a machine learning method (or software) executed by the processor 101. This machine learning method is a machine learning method for learning the adjustment of the measurement operation parameters of the measurement operation, and the CPU of the computer uses the measurement operation parameter data S1 and the measurement time data S2 in the environment in which the measurement device 2 operates. The step of observing as the state variable S representing the current state, the step of acquiring the judgment data D indicating the suitability judgment result of the measurement operation based on the measurement operation parameter of the adjusted measurement operation, and the state variable S and the judgment data D It has a step of learning by associating the measurement time data S2 with the adjustment of the measurement operation parameter of the measurement operation.

In the following second to fourth embodiments, the measurement operation parameter adjusting device 1 according to the first embodiment includes a plurality of devices including a cloud server, a host computer, a fog computer, and an edge computer (robot controller, control device, etc.). An embodiment in which they are connected to each other via a wired / wireless network will be described. As illustrated in FIG. 6, in the following second to fourth embodiments, a layer including a cloud server 6 and the like, a layer including a fog computer 7 and the like, and an edge in a state where each of the plurality of devices is connected to a network. It is assumed that a system is logically divided into three layers including a computer 8 (robot controller, control device, etc. included in cell 9) and the like. In such a system, the measurement operation parameter adjusting device 1 can be mounted on any of the cloud server 6, the fog computer 7, and the edge computer 8, and a network is formed between each of the plurality of devices. The learning data is shared with each other for distributed learning, the generated learning model is collected in the fog computer 7 or the cloud server 6 for large-scale analysis, and the generated learning model is mutually reused. Etc. can be done. In the system illustrated in FIG. 6, a plurality of cells 9 are provided in each of the factories in each region, and each cell 9 is set in a predetermined unit (factory unit, a plurality of factory units of the same manufacturer, etc.) in an upper layer fog computer 7. Manages. Then, the data collected and analyzed by these fog computers 7 can be further collected and analyzed by the cloud server 6 in the upper layer, and the information obtained as a result can be utilized for the control of each edge computer and the like. ..

FIG. 7 shows a system 170 according to a second embodiment including the measurement operation parameter adjusting device 1. The system 170 includes at least one measurement operation parameter adjusting device 1 mounted as a part of a computer such as an edge computer, a fog computer, a host computer, or a cloud server, and a plurality of measuring devices to be adjusted for the measurement operation parameters. 2 and a wired / wireless network 172 that connects the measurement operation parameter adjusting device 1 and the measuring device 2 to each other.

In the system 170 having the above configuration, the measurement operation parameter adjusting device 1 including the machine learning device 100 adjusts the measurement operation parameters of the measurement operation with respect to the time required for the measurement operation by using the learning result of the learning unit 110. It can be calculated automatically and accurately for each measuring device 2. Further, the machine learning device 100 of the measurement operation parameter adjusting device 1 is based on the state variables S and the determination data D obtained for each of the plurality of measuring devices 2, and the measurement operation parameters of the measurement operation common to all the measuring devices 2. It can be configured to learn the adjustment of and share the learning result in the operation of all the measuring devices 2. Therefore, according to the system 170, it is possible to improve the speed and reliability of learning the adjustment of the measurement operation parameters of the measurement operation by inputting a more diverse data set (including the state variable S and the determination data D).

FIG. 8 shows a system 170 according to a third embodiment including the measurement operation parameter adjusting device 1. The system 170 controls at least one machine learning device 100'mounted on a computer 5 such as a cell computer, a fog computer, a host computer, or a cloud server, and a machine 3 on which the measuring device 2 is installed. It includes at least one measurement operation parameter adjusting device 1 mounted as a device (edge computer), and a wired / wireless network 172 that connects the computer 5 and the measurement operation parameter adjusting device 1 to each other.

In the system 170 having the above configuration, the computer 5 provided with the machine learning device 100'from the measurement operation parameter adjusting device 1 (control device) that controls each machine 3 to the machine learning device included in the measuring operation parameter adjusting device 1. The learning model obtained as a result of machine learning by 100 is acquired. Then, the machine learning device 100'provided by the computer 5 generates and generates a newly optimized or efficient learning model by performing knowledge optimization and efficiency improvement processing based on these plurality of learning models. The learned learning model is distributed to the measurement operation parameter adjusting device 1 that controls each machine 3 (measuring device 2).

An example of optimizing or improving the efficiency of the learning model performed by the machine learning device 100'is to generate a distillation model based on a plurality of learning models acquired from each measurement operation parameter adjusting device 1. In this case, the machine learning device 100'according to the present embodiment creates input data to be input to the learning model, and uses the output obtained as a result of inputting the input model to each learning model. A new learning model (distillation model) is generated by learning from. The distillation model thus generated is more suitable for distribution to other devices via an external storage medium, network, or the like, as described above.

As another example of optimizing or improving the efficiency of the learning model performed by the machine learning device 100', each learning for input data is performed in the process of distilling a plurality of learning models acquired from each measurement operation parameter adjusting device 1. Analyzing the distribution of the output of the model by a general statistical method, extracting outliers of the set of input data and output data, and performing distillation using the set of input data and output data excluding the outliers. Is also possible. By going through such a process, an exceptional estimation result is excluded from the set of input data and output data obtained from each training model, and the set of input data and output data excluding the exceptional estimation result is obtained. Can be used to generate a distillation model. The distillation model generated in this way is general-purpose for the machine 3 (measuring device 2) controlled by the measuring operation parameter adjusting device 1 from the learning model generated by the plurality of measuring operation parameter adjusting devices 1. A distillation model can be generated.
In addition, it is possible to appropriately introduce other general learning model optimization or efficiency methods (such as analyzing each learning model and optimizing the hyper parameters of the learning model based on the analysis results). is there.

In the system according to the present embodiment, for example, the machine learning device 100'is arranged on the computer 5 as a fog computer provided for managing a plurality of measurement operation parameter adjusting devices 1 as an edge computer, and each measurement is performed. The learning models generated by the operation parameter adjusting device 1 are aggregated and stored on a fog computer, optimized or streamlined based on the plurality of stored learning models, and then optimized or streamlined. The learning model can be redistributed to each measurement operation parameter adjusting device 1 as needed.

Further, in the system according to the present embodiment, for example, a learning model that is aggregated and stored on the computer 5 as a fog computer, or a learning model that is optimized or streamlined on the fog computer is stored in a higher-level host computer. Collect on a cloud server and use these learning models to apply them to intellectual work at factories and manufacturers of measuring devices 2 (construction and redistribution of more general-purpose learning models on higher-level servers, analysis results of learning models) Support for maintenance work of the measuring device 2 based on the above, analysis of the performance of each measuring device 2, application to the development of a new machine, etc.) can be performed.

FIG. 9 is a schematic hardware configuration diagram of the computer 5 shown in FIG.
The CPU 511 included in the computer 5 is a processor that controls the computer 5 as a whole. The CPU 511 reads the system program stored in the ROM 512 via the bus 520, and controls the entire computer 5 according to the system program. Temporary calculation data, various data input by the operator via the input device 531, and the like are temporarily stored in the RAM 513.

The non-volatile memory 514 is composed of, for example, a memory backed up by a battery (not shown), an SSD (Solid State Drive), or the like, and the storage state is maintained even when the power of the computer 5 is turned off. The non-volatile memory 514 includes a setting area for storing setting information related to the operation of the computer 5, data input from the input device 531, a learning model acquired from each measurement operation parameter adjusting device 1, and an external storage (not shown). Data and the like read via the device and network are stored. The programs and various data stored in the non-volatile memory 514 may be expanded in the RAM 513 at the time of execution / use. Further, a system program including a known analysis program for analyzing various data is written in the ROM 512 in advance.

The computer 5 is connected to the network 172 via the interface 516. At least one measurement operation parameter adjusting device 1 and another computer are connected to the network 172, and data is exchanged with and from the computer 5.

On the display device 530, each data read on the memory, data obtained as a result of executing the program, etc. are output and displayed via the interface 517. Further, the input device 531 composed of a keyboard, a pointing device, and the like passes commands, data, and the like based on operations by the operator to the CPU 511 via the interface 518.
The machine learning device 100 has the same hardware configuration as that described with reference to FIG. 1 except that it is used for optimizing or improving the efficiency of the learning model in cooperation with the CPU 511 of the computer 5.

FIG. 10 shows a system 170 according to a fourth embodiment including the measurement operation parameter adjusting device 1. In the system 170, at least one measurement operation parameter adjusting device 1 mounted as a control device (edge computer) for controlling the machine 3 on which the measuring device 2 is installed, and a plurality of other measuring devices 2 are installed. It includes a control device 4 that controls the machine 3 and a wired / wireless network 172 that connects them to each other.

In the system 170 having the above configuration, the measurement operation parameter adjusting device 1 including the machine learning device 100 includes state data, determination data, and (machine learning device 100) acquired from the machine 3 (measuring device 2) to be controlled. (Not) Machine learning is performed based on the state data and determination data acquired from the other control device 4, and a learning model is generated. The learning model generated in this way is used for adjusting the measurement operation parameters in the measurement operation of the machine (measurement device 2) controlled by itself, and is also used for another machine 3 (measurement device 3) that does not have the machine learning device 100. It is also used for adjusting measurement operation parameters in the other machine 3 (measuring device 2) in response to a request from the control device 4 for controlling 2). Further, when the measurement operation parameter adjusting device 1 newly provided with the machine learning device 100 before the learning model is generated is introduced, the measurement operation parameter adjusting device 1 provided with the learning model via the network 172 It is also possible to acquire and use a learning model.

In the system according to the present embodiment, data and a learning model used for learning can be shared and utilized among a plurality of measurement operation parameter adjusting devices 1 and control devices 4 as a so-called edge computer, so that machine learning can be performed. Improvement of efficiency and reduction of cost for machine learning (introduction of machine learning device 100 to only one control device (measurement operation parameter adjustment device 1), sharing with other control devices 4, etc.) can do.

Although the embodiments of the present invention have been described above, the present invention is not limited to the examples of the above-described embodiments, and can be implemented in various embodiments by making appropriate changes.

For example, the learning algorithm and the calculation algorithm executed by the machine learning device 100, the control algorithm executed by the measurement operation parameter adjusting device 1, and the like are not limited to those described above, and various algorithms can be adopted.

Further, in the above-described embodiment, the measurement operation parameter adjusting device 1 and the machine learning device 100 are described as devices having different CPUs, but the machine learning device 100 is stored in the CPU 11 included in the measurement operation parameter adjusting device 1 and the ROM 12. It may be realized by the system program to be implemented.

1 Measurement operation parameter adjustment device 2 Measurement device 3 Machine 4 Control device 5 Computer 6 Cloud server 7 Fog computer 8 Edge computer 9 Cell 11 CPU
12 ROM
13 RAM
14 Non-volatile memory 17, 18, 19 interface 20 Bus 21 Interface 70 Display device 71 Input device 100 Machine learning device 101 Processor 102 ROM
103 RAM
104 Non-volatile memory 106 State observation unit 108 Judgment data acquisition unit 109 Label data acquisition unit 110 Learning unit 112 Reward calculation unit 114 Value function update unit 122 Decision-making unit 170 System 172 Network 511 CPU
512 ROM
513 RAM
514 Non-volatile memory 516, 516,517 Interface 520 Bus 530 Display device 531 Input device

Claims (14)

It is a measurement operation parameter adjustment device that adjusts the measurement operation parameters of the measurement operation executed by the measurement device that measures the installation position of the object to be measured.
The measurement operation parameter data indicating the measurement operation parameter of the measurement operation and the measurement time data indicating the time required for the measurement operation are observed as state variables representing the current state of the environment, and the measurement operation is based on the state variable. Equipped with a machine learning device that makes learning and decisions using a learning model that models parameter adjustment .
The machine learning device
A state observation unit that observes measurement operation parameter data indicating the measurement operation parameters of the measurement operation and measurement time data indicating the time required for the measurement operation as state variables representing the current state of the environment.
A determination data acquisition unit that acquires measurement operation determination data for determining the suitability of a measurement operation executed based on the measurement operation parameters of the measurement operation as determination data indicating the suitability determination result of the measurement operation.
Using said decision data and the state variables, time required for the measuring operation, and a learning unit configured to generate the learning model trained in association with adjustment of the measuring operation parameters of the measuring operation, Ru provided with,
Measurement operation parameter adjustment device. The state observing unit further observes the variety data indicating the variety information of the measurement object as a state variable representing the current state of the environment.
The learning unit learns by associating the time required for the measurement operation, the type information of the measurement object, and the adjustment of the measurement operation parameters of the measurement operation.
The measurement operation parameter adjusting device according to claim 1 . The learning unit
The remuneration calculation unit that obtains the remuneration related to the suitability judgment result,
Using the reward, a value function update unit that updates a function representing the value of the adjustment action of the measurement action parameter of the measurement action with respect to the time required for the measurement action, and
With
The reward calculation unit gives a higher reward as the time required for the measurement operation is shorter, and when the measurement object and the measuring device are damaged, the measurement is performed as the time required for the measurement operation is slower. If the object moves, give a low reward,
The measurement operation parameter adjusting device according to claim 1 or 2 . The learning unit calculates the state variable and the determination data in a multi-layer structure.
The measurement operation parameter adjusting device according to claim 1 or 2 . The machine learning device
A decision-making unit that determines adjustment of measurement operation parameters of measurement operation based on the state variables observed by the state observation unit and the learning model is further provided.
The measurement operation parameter adjusting device according to claim 1 . The machine learning device exists in the cloud server.
The measurement operation parameter adjusting device according to any one of claims 1 to 5 . The measurement operation parameter data indicating the measurement operation parameters of the measurement operation executed by the measuring device for measuring the installation position of the measurement object and the measurement time data indicating the time required for the measurement operation are state variables representing the current state of the environment. Based on the state variable, learning or making a decision using a learning model that models the adjustment of the measurement operation parameters.
It ’s a machine learning device,
A state observation unit that observes measurement operation parameter data indicating the measurement operation parameters of the measurement operation and measurement time data indicating the time required for the measurement operation as state variables representing the current state of the environment.
A determination data acquisition unit that acquires measurement operation determination data for determining the suitability of a measurement operation executed based on the measurement operation parameters of the measurement operation as determination data indicating the suitability determination result of the measurement operation.
A learning unit that generates the learning model learned by associating the time required for the measurement operation with the adjustment of the measurement operation parameter of the measurement operation using the state variable and the determination data.
A machine learning device equipped with. A decision-making unit that determines adjustment of measurement operation parameters of measurement operation based on the state variables observed by the state observation unit and the learning model.
7. The machine learning device according to claim 7 . A system in which multiple devices are connected to each other via a network.
The plurality of devices include at least the first measurement operation parameter adjusting device according to claim 1.
System . The plurality of devices include a computer equipped with a machine learning device.
The computer acquires at least one learning model generated by learning in the learning unit of the measurement operation parameter adjusting device.
The machine learning device included in the computer optimizes or improves efficiency based on the acquired learning model.
The system according to claim 9 . The plurality of devices include a second measurement operation parameter adjusting device different from the first measurement operation parameter adjusting device.
The learning model by the learning unit included in the first measurement operation parameter adjusting device is shared with the second measurement operation parameter adjusting device.
The system according to claim 9 . The plurality of devices include a control device for controlling the measuring device.
The system according to claim 9, wherein the data related to the measuring device acquired by the control device can be used for learning by the learning unit included in the first measuring operation parameter adjusting device via the network . The measurement operation parameter data indicating the measurement operation parameters of the measurement operation for measuring the installation position of the measurement object and the measurement time data indicating the time required for the measurement operation are observed as state variables representing the current state of the environment, and the state is observed. Learning the adjustment of measurement operation parameters by executing the step of learning or making a decision using the learning model that models the adjustment of the measurement operation parameters based on the variables with the measuring device that measures the installation position of the measurement object. Or a way to make decisions
The step of learning or making a decision using the learning model is
A step of observing measurement operation parameter data indicating the measurement operation parameters of the measurement operation and measurement time data indicating the time required for the measurement operation as state variables representing the current state of the environment.
A step of acquiring measurement operation determination data for determining the suitability of a measurement operation executed based on the measurement operation parameters of the measurement operation as determination data indicating the suitability determination result of the measurement operation.
A learning step of learning by associating the time required for the measurement operation with the adjustment of the measurement operation parameter of the measurement operation using the state variable and the determination data.
A method of learning or making decisions about the adjustment of measurement operating parameters . The step of learning or making a decision using the learning model is
Further performing a step of determining the adjustment of the measurement motion parameter of the measurement motion based on the observed state variable and the learning result learned by associating the time required for the measurement motion and the adjustment of the measurement motion parameter of the measurement motion.
The method for learning or making a decision on adjustment of measurement operation parameters according to claim 13 .

Images