JP2018151959A - Determining apparatus, determining program and learning method - Google Patents

Abstract

An object of the present invention is to determine whether an action or state of an object controlled by artificial intelligence is an action or state that may have undesirable consequences. According to one aspect of the present invention, a determination device includes an acquisition unit and a determination unit. The acquisition unit is controlled by the first artificial intelligence in which the first learning result obtained through the first machine learning including causing the learning artificial intelligence to control the operation or state of the learning target is set. Get input data representing the observation results of the motion or state of the target. The determination unit determines whether or not the input data represents an action requiring attention or a state requiring attention determined based on an action or state of a learning target in a period in which the first machine learning is executed. [Selection] Figure 1

Description

  The present invention relates to a technique for determining the validity of an output of artificial intelligence.

  Machine learning is performed to cause the artificial intelligence to acquire intellectual ability such as classifying data. For example, Patent Document 1 discloses a technique for determining pass / fail of an inspection object from an inspection image using a neural network.

  Machine learning may be performed for the purpose of acquiring the ability to control the motion or state of the object (control ability). By such machine learning, the artificial intelligence may acquire a control ability to perform an operation of, for example, grasping a work with a robot hand (or a pick-and-place device) and moving it from point A to point B. . Thereby, even if a human does not code the control program of a robot hand one by one, artificial intelligence can perform the operation concerning a robot hand.

  However, the control capability acquired by machine learning does not always produce desirable results. That is, the target may perform an unintended operation (for example, a dangerous operation such as a collision) due to a malfunction such as runaway, or may fall into an unintended state (for example, a dangerous state such as a high temperature exceeding the recommended temperature range). there is a possibility.

  In general, it is virtually impossible for a designer to code a decision program with an exhaustive definition of all the actions or states of interest that produce undesirable results. In particular, when the artificial intelligence is made to control the operation or state of the object, the contents controlled by the artificial intelligence cannot be predicted by the human programmer, and therefore the determination program cannot be coded by the human programmer.

  Patent Document 2 discloses a technique for evaluating the likelihood of an output node value of a neural network based on whether or not the input node value of the neural network matches or is similar to the input node value of learning data. In this document, the probability of output is evaluated depending on whether the input to the neural network is data used for learning or unlearned data. However, if the data used at the time of learning is input, the target controlled by the output of the neural network does not always give a desirable result.

JP 2004-191112 A JP-A-5-225163

  It is an object of the present invention to determine whether an action or state of an object controlled by artificial intelligence is an action or state that can have undesirable consequences.

  According to the first aspect of the present invention, the determination device includes an acquisition unit and a determination unit. The acquisition unit is controlled by the first artificial intelligence in which the first learning result obtained through the first machine learning including causing the learning artificial intelligence to control the operation or state of the learning target is set. Get input data representing the observation results of the motion or state of the target. The determination unit determines whether or not the input data represents an action requiring attention or a state requiring attention determined based on an action or state of a learning target in a period in which the first machine learning is executed. Therefore, according to this aspect, generation | occurrence | production of an undesirable result can be determined from the observation result of operation | movement or a state of object.

  According to the second aspect of the present invention, the determination unit uses the observation data representing the observation result of the learning operation or state as the learning data in the period in which the first machine learning is executed. Using the second learning result obtained through learning, it is determined whether or not the input data represents an action requiring attention or a condition requiring attention. Therefore, according to this aspect, the learning result of machine learning can be used to predict the occurrence of an undesirable result.

  According to the third aspect of the present invention, the determination unit gives input data to the second artificial intelligence in which the second learning result is set, and whether or not the input data represents the action requiring attention or the condition requiring attention. To determine. Therefore, according to this aspect, it is possible to cause the artificial intelligence to predict the occurrence of an undesirable result from the observation result of the motion or state of the target.

  According to the fourth aspect of the present invention, the determination device further includes an alarm unit that issues an alarm when the input data represents a cautionary action or a cautionary state. Therefore, according to this aspect, an alarm can be issued when an undesirable result is predicted.

  According to the fifth aspect of the present invention, the determination apparatus further includes an intervention unit that intervenes in the control to the target action or state when the input data represents the action requiring attention or the condition requiring attention. Thus, according to this aspect, it is possible to intervene in the control of a subject's action or state when occurrence of an undesirable result is predicted.

  According to the sixth aspect of the present invention, in the learning method, the first learning result obtained through the first machine learning including causing the learning artificial intelligence to control the operation or state of the learning target is obtained. Obtaining input data representing observation results of the motion or state of the object controlled by the set first artificial intelligence. Further, the learning method includes causing the second learning artificial intelligence to determine whether or not the input data represents an action requiring attention or a condition requiring attention determined based on the first learning data. Including learning. According to this aspect, it is possible to perform machine learning for predicting the occurrence of an undesirable result from the observation result of the target action or state.

  According to the seventh aspect of the present invention, the first observation data is learned during a period in which the learning achievement degree of the first learning artificial intelligence is equal to or less than a threshold in the period in which the first machine learning is executed. Represents the observation result of the motion or state of the target. Therefore, according to this aspect, it is possible to learn the action requiring attention and the action or condition corresponding to the condition requiring attention based on the action or the state having a high possibility of causing an undesirable result.

  According to the eighth aspect of the present invention, the first observation data is the first learning result obtained through the first learning artificial intelligence or the first machine learning after the completion of the first machine learning. Includes third observation data that does not coincide with the second observation data representing the observation result of the action or state of the target controlled by the first artificial intelligence set. The first learning data is created based on the third observation data. Therefore, according to this aspect, it is possible to learn the action or state corresponding to the action requiring attention and the condition requiring attention, excluding the action or state estimated to be appropriate with high probability.

  According to the ninth aspect of the present invention, the second machine learning is performed using the first learning data and the second learning data created based on the second observation data. The first learning data includes first teacher data indicating that the third observation data corresponds to the action requiring attention or the condition requiring attention. The second learning data includes second teacher data indicating that the second observation data does not correspond to the action requiring attention or the condition requiring attention. Therefore, according to this aspect, it is possible to learn more accurately the action or state corresponding to the action requiring attention or the condition requiring attention.

  In accordance with the present invention, it is possible to determine whether an action or state of an object controlled by artificial intelligence is an action or state that may have undesirable results.

1 is a block diagram illustrating a determination device according to a first embodiment. The block diagram which illustrates the control system containing the determination apparatus of FIG. The block diagram which illustrates the machine learning system for learning the capability provided to the control apparatus and determination apparatus of FIG. The block diagram of the service provision system which provides the machine learning service of the control apparatus and determination apparatus of FIG. 2 according to the request | requirement from a user. 5 is a flowchart illustrating the operation of the service providing system in FIG. 4. Explanatory drawing of operation | movement of the determination apparatus of FIG.

  Hereinafter, embodiments will be described with reference to the drawings. Hereinafter, elements that are the same as or similar to elements already described are denoted by the same or similar reference numerals, and redundant descriptions are basically omitted.

(First embodiment)
As illustrated in FIG. 1, the determination apparatus 100 according to the first embodiment includes an acquisition unit 101, a determination unit 102, an alarm unit 104, an intervention unit 105, and a storage unit 106. The determination apparatus 100 acquires input data representing observation results of a target operation or state described later. Here, the target is not limited to a direct control target, and may include an indirect control target that is affected by the operation or state of the direct control target. For example, the object may include a robot hand and a work that is grasped by the robot hand. Also, an operation or state can include one or both of the operation and the state. Operations and states may not be clearly distinguished.

  The determination apparatus 100 determines whether or not the input data represents a caution operation or a caution state described later. Furthermore, the determination apparatus 100 may issue an alarm when the input data indicates a cautionary action or a cautionary state, or may intervene in control of the target action or state.

  The acquisition unit 101 is, for example, an interface (for example, an input port or a communication port) for connecting to an external device, and can be appropriately configured according to the external device to be connected. The acquisition unit 101 acquires input data. The acquisition unit 101 may send the input data to the determination unit 102 as it is, or may send the input data to the determination unit 102 after performing necessary processing (such as data format conversion).

  The input data represents the observation result of the operation or state of the object controlled by the artificial intelligence described later. The subject can be a variety of controllable devices such as industrial, surgical, entertainment or other robots, chemical plants, engines for cars or other machinery, therapeutic devices such as nebulizers, building air conditioning systems, etc. . Then, for example, when the target is a robot hand, the input data is obtained based on observations of the locus of movement of the robot hand, the movement of the motor, the value of current applied to the motor, and the like. For example, when the target is a chemical plant, it is obtained based on observation of the state of combustion or other chemical reaction in the reaction tank (for example, the temperature of the target or its surroundings, humidity). In addition, for example, when the target is an engine for an automobile or other machine, it is obtained based on observations of operation information of the automobile or the engine itself, the number of revolutions of the engine, the amount of fuel to be supplied, the temperature of combustion, and the like. Further, for example, when the target is a nebulizer, it is obtained based on the observation with the vibration frequency of the vibrator. Further, for example, when the target is a building air conditioning system, it may be obtained based on observation of power consumption or the like.

  For observation of the movement or state of the object, for example, an image sensor (camera), microphone, rotation speed sensor, acceleration sensor, angular velocity sensor, current sensor, temperature sensor, humidity sensor, vibration sensor, noise sensor, operation time sensor, power consumption Various sensors such as sensors can be used. The acquisition unit 101 may acquire input data directly from an observation device such as the sensor, or may acquire input data stored in a storage (not shown) by the observation device.

  The action or state to be observed may be selected that can be used to determine whether there is a risk of undesirable results. For example, when it is determined whether there is a possibility that the target will operate in a range exceeding a predetermined durable temperature, durable current, or durable voltage, the temperature is applied to the subject or its surrounding temperature and the subject. Current or voltage may be observed.

  The determination unit 102 may include, for example, a CPU (Central Processing Unit) that is a hardware processor, a RAM (Random Access Memory), a ROM (Read Only Memory), and the like. Specifically, the CPU expands the program stored in the storage unit 106 in the RAM. Then, when the CPU interprets and executes the program, the determination unit 102 performs an operation (information processing) described below.

  The determination unit 102 receives input data from the acquisition unit 101, and determines whether or not the input data represents an action requiring attention or a state requiring attention. The determination unit 102 notifies the alarm unit 104 and the intervention unit 105 of the determination result.

  The action requiring attention or the condition requiring attention is determined based on the action or state of the object for learning in the period in which the machine learning for acquiring the ability to control the action or state of the object (control ability) is executed. Here, the machine learning includes causing the learning artificial intelligence to control the operation or state of the learning target.

  Note that the artificial intelligence for learning may be the same as or different from the artificial intelligence that actually controls the object (that is, used during operation). The artificial intelligence that actually controls the object only needs to have a learning result obtained through this machine learning.

  The target for learning may be the same as or different from the target whose operation or state is actually controlled by artificial intelligence (that is, used during operation). Specifically, the learning target may be a real machine of the same type as the target that is actually controlled, or may be a simulation model. That is, machine learning may be executed on a simulation.

  In general, if it is highly likely that the learning target will perform a dangerous motion or fall into a dangerous state during machine learning, and the damage that can be caused by it is expected to be large, machine learning should be performed. It is preferable to execute it on a simulation. On the other hand, learning is performed when safety measures are taken against dangerous movements (for example, the environment of use is adjusted so that persons or objects unrelated to the operation of the robot hand do not enter the movable range of the robot hand). When there is a low possibility of failure or breakage of a target for use, machine learning can be performed using an actual machine. In addition, when there is an individual difference in the target or its use environment, and it is necessary to perform precise control for each target, an actual machine used as a learning target can be diverted as it is during operation.

  Machine learning to acquire control ability typically corresponds to reinforcement learning, and during execution (particularly in the initial stage), the learning target is given a randomly determined control value, and dangerous actions (for example, The robot hand moves to the limit of the range of motion), or it falls into various states including dangerous states.

  Artificial intelligence for learning gradually learns to avoid actions or states that produce undesirable results and to direct actions or states that produce desirable results. Therefore, by observing the movement or state of the object for learning during the period during which machine learning is executed (especially in the initial stage), it is possible to grasp the cautionary action or the state of caution that causes an undesirable result. .

  The operation or state to be observed may be selected in advance by a human (designer or operator of the determination apparatus). In this case, the trouble of observing unnecessary data can be saved. Or, as long as machine learning is performed, observe as many different movements or states as possible, and later used by humans or software (which may include artificial intelligence) to determine the actions and conditions that require attention. An action or state may be selected. In this case, it is possible to reduce the cautionary action and missed attention state.

  The determination unit 102 may perform determination using artificial intelligence. Specifically, the determination unit 102 gives input data to an AI (Artificial Intelligence) 103 that has acquired the ability (determination ability) to determine whether or not the input data represents an action requiring attention or a state requiring attention. You may make a judgment. The AI 103 and other AI described later are realized by a processor such as a GPU (Graphics Processing Unit) executing a program expanded in the RAM.

  Alternatively, the determination unit 102 converts the input data (which is actually controlled by artificial intelligence) into observation data representing the observation result of the operation or state of the learning target in the period (particularly the initial stage) in which machine learning is executed. Determination may be made based on whether or not there is a match.

  The AI 103 may be a deep neural network, for example. The AI 103 is set with a learning model having learning parameters (for example, unit bias in the neural network, edge weights between units, etc.) obtained through machine learning (for example, deep learning) to acquire determination ability. The Details of the machine learning will be described later with reference to FIG.

  As described above, the determination unit 102 determines whether or not the observation result of the action or state of the target that is actually controlled by the artificial intelligence represents the action requiring attention or the condition requiring attention. Therefore, as shown in FIG. 6, the determination unit 102 monitors the output from the artificial intelligence 201 in a scene where the artificial intelligence 201 that controls the object 210 is actually used, and the output is a cautionary action or necessary. It can be determined whether or not an attention state is caused. Further, the alarm unit 104 and the intervention unit 105 perform processing to be described later according to the determination result.

  The alarm unit 104 may be an alarm device such as a buzzer or a speaker, or may be an interface for connecting to an external alarm device. A determination result is notified from the determination unit 102. The alarm unit 104 issues an alarm when notified of the determination result that the input data represents an action requiring attention or a condition requiring attention. The alarm is, for example, to visually or auditorily notify a human (target user, administrator, etc.) that there is a problem with the operation or state of the target using an image, light, sound, or the like. Or the operation of the target with respect to the target or a control device that directly or indirectly controls the target (artificial intelligence, a device that generates a control signal of the target based on a control value from the artificial intelligence) You may notify that there is a problem in a state. Specifically, the alarm unit 104 may turn on an icon on the meter panel or prompt the relearning of the AI that is controlling the target. The person who received the warning, or the object or its control device, stops or resets the control of the object, or controls the operation or state of the object by a human or another control device (eg, a manually coded control program). Undesired results can be avoided by entrusting the control device to the control device) or re-learning the AI.

  The intervention unit 105 can include, for example, a CPU, RAM, ROM, etc., which are hardware processors. Specifically, the CPU expands a program stored in the storage unit 106 or another storage unit (for example, a storage unit (not shown) built in the intervention unit 105) in the RAM. When the CPU interprets and executes the program, the intervention unit 105 performs an operation (information processing) described below. Such a program receives, for example, a signal indicating a determination result that the input data represents a cautionary action or a cautionary state from the CPU of the determination unit 102, and a signal for intervening in the control of the target 210 (for example, the target 210 An instruction for outputting a signal to stop) may be described.

  The intervention unit 105 is notified of the determination result from the determination unit 102. When the intervention unit 105 is notified of the determination result that the input data represents the action requiring attention or the condition requiring attention, the intervention unit 105 intervenes in the control to the target action or state. Intervention may be, for example, stopping or resetting control to a subject's action or state, and transferring control to the subject's action or state to a human or another control device (eg, manually coded control). It may be left to a control device that executes the program.

  The alarm by the alarm unit 104 and the intervention by the intervention unit 105 may be performed in stages. Specifically, the risk of the action requiring attention or the condition requiring attention may be evaluated in accordance with the result of the action requiring attention or the condition requiring attention (for example, the aforementioned low reward). If a low-risk action or condition requiring attention is detected, a low-restraint alarm and intervention are performed, and if a high-risk action or condition requiring attention is detected, the restraint action is detected. Strong alerts and interventions may be performed.

  The alarm unit 104 may be deleted from the determination apparatus 100. Specifically, the alarm unit 104 may be incorporated in the target or other device, or may not exist at all. Similarly, the intervention unit 105 may be deleted from the determination apparatus 100. Specifically, the intervention unit 105 may be incorporated in the target or other device, or may not exist at all.

  The storage unit 106 may include, for example, a hard disk drive (HDD: Hard Disk Drive), a solid state drive (SSD), and the like. For example, the storage unit 106 stores a program executed by the CPU of the determination unit 102. Such a program is read by the CPU of the determination unit 102 as necessary, is expanded in the RAM of the determination unit 102, and is executed by the CPU. Further, the storage unit 106 may store a program executed by the CPU of the intervention unit 105.

  The determination apparatus 100 of FIG. 1 can be incorporated into the control system shown in FIG. 2, for example. The control system of FIG. 2 includes a control device 200, a target 210, an observation device 220, and a determination device 100.

  The control device 200 generates a control value for controlling the operation or state of the target 210 based on the input data. The input data represents information that can be referenced to control the operation or state of the object, such as, for example, an image, temperature. The control device 200 sends this control value to the target 210 as output data. For example, when the target 210 is an engine, the control device 200 may be an engine control ECU (Electronic Control Unit). The control device 200 may be, for example, a controller for controlling a robot hand when the target 210 is a robot hand. The control device 200 gives input data to the AI 201 in which a learning result obtained through machine learning for acquiring control ability for the object 210 is set, and obtains output data.

  The target 210 receives a control value from the control device 200 and responds to it. For example, the target 210 injects fuel, changes the rotation speed of the crankshaft, changes the set temperature of the air conditioning, or drives the motor to move the robot hand. The operation or state of the object 210 is observed by the observation device 220.

  The observation device 220 includes various types such as an image sensor (camera), a microphone, a rotation speed sensor, an acceleration sensor, an angular velocity sensor, a current sensor, a temperature sensor, a humidity sensor, a vibration sensor, a noise sensor, an operation time sensor, and a power consumption sensor. Can be included. The observation device 220 observes the operation or state of the object 210 and generates observation data. The observation device 220 sends observation data to the determination device 100 as input data.

  The determination apparatus 100 receives input data from the observation apparatus 220, and determines whether or not the input data represents an action requiring attention or a condition requiring attention. The determination device 100 may issue a warning to the control device 200 or a human or other control device (not shown) based on the determination result, or may intervene in the control of the operation or the state of the target 210. The determination device 100 may be a device independent of the target 210 or may be incorporated in the target 210. Further, the determination device 100 may be a device independent of the control device 200, or may be incorporated in the control device 200.

  FIG. 3 illustrates a machine learning system that performs machine learning (reinforcement learning) for acquiring control capability and machine learning (supervised learning) for acquiring determination capability. In these machine learning, learning data stored in the learning database device 310 is used. In these machine learnings, test data (not shown) stored in the learning database device 310 may be used for evaluation of learning achievement. Further, learning data used for machine learning for obtaining control ability (control ability learning data) and learning data used for machine learning for obtaining judgment ability (determination ability learning data) are different learning databases. It may be stored in the device.

  The first learning data creation device 300 creates control ability learning data corresponding to the input data given to the learning AI 321 and stores it in the learning database device 310. The input data represents information that can be referenced to control the operation or state of the object, such as, for example, an image, temperature.

  The control capability learning device 320 performs machine learning including causing the learning AI 321 to control the operation or state of the learning target 330. Specifically, the control ability learning device 320 reads the control ability learning data from the learning database device 310, gives this to the AI 321 as input data, and gives the output data (directly or indirectly to the learning target 330). Control value). The output data may be, for example, a control value given to an actuator such as a hydraulic cylinder, a motor, or sono laid. When the target 330 is an engine, for example, a command for fuel injection to the engine, a command for increasing or decreasing the rotational speed of the crankshaft, and the like may be used. When the target 330 is a robot hand, for example, a command for increasing or decreasing the supply current to each actuator may be used. The AI 321 learns by acquiring a high reward if the target 330 controlled by the output data has a desired result, and acquiring a low reward if the target 330 has an undesirable result. The result brought about by the object 330 can be evaluated based on observation by the observation device 340, for example. Desirable results are determined depending on the ability or the like desired to be acquired by machine learning, and may be, for example, reduction of time to completion of operation, improvement of fuel consumption, reduction of environmental load, reduction of power consumption, and the like. The control ability learning device 320 performs machine learning (update of learning parameters) of the AI 321 so as to maximize this reward.

  The control capability learning device 320 evaluates the learning achievement level of the learning AI 321 using the test data, and when the learning target is reached, the AI that controls the learning result (learning model) to the target (for example, FIG. 2 AI 201).

  The AI that controls the target may be an AI that is different from the learning AI 321, or the learned AI 321 may be diverted. Further, the control capability learning device 320 and the AI that performs control on the target may be incorporated in the same device (for example, an ECU or a controller of a robot).

  The observation device 340 observes the operation or state of the learning target 330 during a period in which machine learning is executed by the control capability learning device 320, and generates first observation data. The observation device 340 sends the first observation data to the second learning data creation device 350. The first observation data represents an observation result of the operation or state of the target 330 in a period in which the learning achievement of the learning AI 321 is equal to or less than a threshold in the period in which machine learning is performed by the control ability learning device 320. Also good. In general, the higher the learning achievement of AI (the closer the completion of machine learning is), the lower the possibility that an undesirable result will occur. Therefore, in the initial stage of machine learning, an operation that is impossible when the target 330 is operated is performed or falls into a state. This impossible operation or state can be said to be a cautionary action or a cautionary state during operation of the target 330. For this reason, it is possible to learn the cautionary action and the cautionary state by obtaining this impossible action or state, which is performed in the initial stage of the machine stage, as first observation data and learning with the determination ability learning device 360 described later. I may be able to do it.

  The first observation data is a period in which the learning achievement degree of the learning AI 321 is equal to or higher than the first threshold and equal to or lower than the second threshold in the period when the machine learning is executed by the control ability learning device 320. It may be an observation result of the operation or state of the target 330 in FIG.

  The second learning data creation device 350 creates determination ability learning data that is learning data used for machine learning for acquiring determination ability. Specifically, the second learning data creation device 350 receives the first observation data from the observation device 340.

  The second learning data creation device 350 creates determination ability learning data based on the first observation data. The determination ability learning data includes at least first learning data to be described later. The determination ability learning data may or may not include second learning data described later.

  The determination capability learning device 360 may include a processor such as a CPU, a storage unit that stores a program executed by the processor, and a RAM for developing the program. The CPU expands the program stored in the storage unit in the RAM, interprets and executes it, and performs the operations described below.

  The determination ability learning device 360 uses the determination ability learning data to cause the learning AI 361 to determine whether or not the input data represents an action requiring attention or a state requiring attention determined based on the determination ability learning data. Including machine learning. Specifically, the determination capability learning device 360 gives input data included in the learning data to the AI 361, and obtains output data (determination result). The determination capability learning device 360 performs machine learning (update of learning parameters) of the AI 361 so that the output data approaches the teacher data included in the learning data.

  The determination capability learning device 360 evaluates the learning achievement level of the learning AI 361 using test data, and sets the learning result (learning model) in the AI 103 of the determination device 100 when the learning target is reached.

  The AI 103 may be an AI that is separate from the learning AI 361, or the learned AI 361 may be diverted. Further, the determination capability learning device 360 may be incorporated in the determination device 100.

  The first learning data includes, for example, first observation data (input data) and a value (teacher data) indicating that the operation or state represented by the first observation data corresponds to a caution operation or a caution state. It may be a pair.

  Alternatively, the first learning data includes the third observation data (input data) that is a part or all of the first observation data, and the operation or state represented by the third observation data. It may be a pair with a value (teacher data) indicating that it corresponds to a state. Here, the third observation data refers to the first observation data that does not match the second observation data. The second observation data is an observation of the action or state of the target controlled by the AI 321 after completion of the machine learning by the control ability learning device 320 or the AI to which the learning result obtained through the machine learning is set. Represents the result. In general, since the motion or state represented by the second observation data can be estimated with high probability as an appropriate motion or state, it may be excluded from the first learning data. On the other hand, second learning data may be created based on the second observation data. The second learning data includes, for example, second observation data (input data) and a value (teacher data) indicating that the operation or state represented by the second observation data does not correspond to a caution operation or a caution state. It may be a pair.

  FIG. 4 illustrates a service providing system that provides a machine learning service for acquiring control ability and determination ability in response to a request from a user. The service providing system of FIG. 4 includes a first learning data creation device 300, a learning database device 310, a control ability learning device 320, a second learning data creation device 350, a judgment ability learning device 360, and a user. A terminal 400 and a learning request apparatus 410 are included. Each device is connected to each other via a network NW.

  The apparatus configuration of the service providing system in FIG. 4 is merely an example. That is, some or all of the devices shown in FIG. 4 may be integrated into one device, or the functions of the device shown in FIG. 4 may be distributed to a plurality of devices. 4 may be deleted or changed, or a device not shown in FIG. 4 may be added.

  4 may basically include an input / output unit, a processor such as a CPU, a storage unit that stores a program executed by the processor, and a RAM that expands the program. The CPU expands the program stored in the storage unit in the RAM, interprets and executes it, and performs the operations described below.

  Here, the input / output unit is, for example, an interface (for example, an input / output port or a communication port) for connecting to an external device (via a network), and can be appropriately configured according to the external device to be connected. The input / output unit receives input data to be supplied to the CPU from an external device, and sends output data from the CPU to the external device.

  As described above, the first learning data creation device 300 creates control ability learning data and stores it in the learning database device 310. Data serving as a base for creating control ability learning data may be included in the input data from the learning request apparatus 410.

  The control ability learning device 320 receives a learning start command from the learning request device 410. Furthermore, the control capability learning device 320 may receive specification data that defines the control capability desired to be acquired by machine learning, for example. The control capability learning device 320 may set a learning target of the learning AI 321 or set a reward for reinforcement learning based on the specification data.

  The learning database device 310 stores control ability learning data used for machine learning for acquiring control ability, and judgment ability learning data used for machine learning for obtaining judgment ability. The control capability learning data is created by the first learning data creation device 300 and read by the control capability learning device 320 as necessary. The determination capability learning data is created by the second learning data creation device 350 and read by the determination capability learning device 360 as necessary. The determination capability learning data includes the first learning data described above, and may optionally include second learning data.

  The second learning data creation device 350 creates determination ability learning data based on the observation data and stores it in the learning database device 310. The observation data serving as a base for creating the judgment ability learning data may be obtained directly from an observation device not shown in FIG. 4 or may be included in the input data from the learning request device 410. Good.

  The determination ability learning device 360 receives a learning start command from the learning request device 410. Further, the determination capability learning device 360 may receive specification data from the learning request device 410. The determination capability learning device 360 may set a learning target of the learning AI 361 based on the specification data.

  The user terminal 400 requests the service providing system to provide machine learning suitable for the user's purpose. For example, the user terminal 400 receives specification data and a learning start command, and sends them to the learning request device 410. The specification data may include a value indicating the target structure, application, purpose, load bearing performance, or the like.

  The learning request device 410 receives specification data and a learning start command from the user terminal 400 as a learning request. The learning request apparatus 410 sends the specification data and the learning start command to the control ability learning apparatus 320 and the determination ability learning apparatus 360.

  FIG. 5 illustrates the operation of the service providing system of FIG. When the user inputs specification data and a learning start command to the user terminal 400, the operation in FIG.

  First, the learning request apparatus 410 receives specification data and a learning start command from the user terminal 400 as a learning request (step S501). Using this learning request as a trigger, the first learning data creation device 300 creates control ability learning data and stores it in the learning database device 310 (step S502).

  The control capability learning device 320 performs machine learning of the AI 321 using the control capability learning data created in step S502 (step S503). The learning result in step S503 can be sent to the user terminal 400 and set to the AI used by the user.

  Based on the observation data of the target action or state in step S503, the second learning data creation device 350 creates determination ability learning data and stores it in the learning database device 310 (step S504).

  The determination capability learning device 360 performs machine learning of the AI 361 using the determination capability learning data created in step S504 (step S505). The learning result in step S505 can be sent to the user terminal 400 and set to the AI used by the user.

  As described above, the determination apparatus according to the first embodiment determines whether or not the input data representing the observation result of the action or state of the target controlled by the artificial intelligence represents the action requiring attention or the condition requiring attention. Determine. Since these action requiring attention or condition requiring attention are determined based on the action or state of the object for learning at the time of machine learning for the artificial intelligence, it can be estimated that there is a high possibility of causing an undesirable result. Therefore, according to this determination apparatus, generation | occurrence | production of an undesired result can be estimated from the observation result of operation | movement or a state of object.

  The above-described embodiments are merely specific examples for helping understanding of the concept of the present invention, and are not intended to limit the scope of the present invention. The embodiment can add, delete, or convert various components without departing from the gist of the present invention.

  The various functional units described in the above embodiments may be realized by using a circuit. The circuit may be a dedicated circuit that realizes a specific function, or may be a general-purpose circuit such as a processor.

  At least a part of the processing of each of the above embodiments can also be realized by using a general-purpose computer as basic hardware. A program for realizing the above processing may be provided by being stored in a computer-readable recording medium. The program is stored in the recording medium as an installable file or an executable file. Examples of the recording medium include a magnetic disk, an optical disk (CD-ROM, CD-R, DVD, etc.), a magneto-optical disk (MO, etc.), and a semiconductor memory. The recording medium may be any recording medium as long as it can store the program and can be read by the computer. The program for realizing the above processing may be stored on a computer (server) connected to a network such as the Internet and downloaded to the computer (client) via the network.

A part or all of each of the embodiments described above can be described as shown in the following supplementary notes in addition to the claims, but is not limited thereto.
(Appendix 1)
Memory,
A processor connected to the memory,
The processor is
(A) The first learning result obtained through the first machine learning including causing the learning artificial intelligence to control the operation or state of the learning target is controlled by the set first artificial intelligence. Obtain input data that represents the observation results of the motion or state of the target
(B) Determining whether or not the input data represents an action requiring attention or a condition requiring attention determined based on the action or state of the learning target in a period in which the first machine learning is executed. A determination device configured as follows.

(Appendix 2)
A learning method performed by a processor,
The learning method is:
A first representing an observation result of the learning target operation or state in a period in which the first machine learning including execution of the first learning artificial intelligence to control the learning target operation or state is performed. Obtaining first learning data created based on the observation data of
Performing a second machine learning including causing the second learning artificial intelligence to determine whether or not the input data represents an action requiring attention or a condition requiring attention determined based on the first learning data. A learning method comprising:

DESCRIPTION OF SYMBOLS 100 ... Determination apparatus 101 ... Acquisition part 102 ... Determination part 103,201,321,361 ... AI
104 ... alarm unit 105 ... intervention unit 106 ... storage unit 200 ... control device 210,330 ... target 220,340 ... observation device 300 ... first learning data creation device 310 ... Learning database device 320 ... Control capability learning device 350 ... Second learning data creation device 360 ... Judgment capability learning device 400 ... User terminal 410 ... Learning request device

Claims (10)

The first learning result obtained through the first machine learning including causing the learning artificial intelligence to control the movement or state of the learning target is set for the target controlled by the first artificial intelligence. An acquisition unit for acquiring input data representing observation results of the operation or state;
A determination unit that determines whether or not the input data represents an action requiring attention or a state requiring attention determined based on the action or state of the learning target in a period in which the first machine learning is performed; A determination device.   The determination unit uses a second machine learning obtained through a second machine learning using observation data representing an observation result of an operation or a state of the learning target in a period in which the first machine learning is executed as learning data. The determination apparatus according to claim 1, wherein a determination result is used to determine whether the input data represents the action requiring attention or the condition requiring attention.   The determination unit gives the input data to the second artificial intelligence in which the second learning result is set, and determines whether the input data represents the action requiring attention or the condition requiring attention. The determination apparatus according to claim 2.   The determination apparatus according to claim 1, further comprising an alarm unit that issues an alarm when the input data represents the action requiring attention or the condition requiring attention.   The determination apparatus according to claim 4, further comprising an intervention unit that intervenes in the control to the operation or state of the target when the input data represents the operation requiring attention or the state requiring attention. Computer
The first learning result obtained through the first machine learning including causing the learning artificial intelligence to control the movement or state of the learning target is set for the target controlled by the first artificial intelligence. Means for obtaining input data representing observation results of movements or states;
A function for determining whether or not the input data represents an action requiring attention or a condition requiring attention determined based on the action or state of the learning target in a period in which the first machine learning is executed; Judgment program for. A first representing an observation result of the learning target operation or state in a period in which the first machine learning including execution of the first learning artificial intelligence to control the learning target operation or state is performed. Obtaining first learning data created based on the observation data of
Performing a second machine learning including causing the second learning artificial intelligence to determine whether or not the input data represents an action requiring attention or a condition requiring attention determined based on the first learning data. A learning method comprising:   The first observation data is an operation or state of the learning target in a period in which the learning achievement of the first artificial intelligence for learning is less than or equal to a threshold in the period in which the first machine learning is executed. The learning method according to claim 7, wherein the learning result is expressed. In the first observation data, the first learning result obtained through the first learning artificial intelligence or the first machine learning after the completion of the first machine learning is set. Including third observation data that does not match the second observation data representing the observation result of the action or state of the object controlled by the artificial intelligence;
The first learning data is created based on the third observation data.
The learning method according to claim 7. The second machine learning is performed using the first learning data and the second learning data created based on the second observation data,
The first learning data includes first teacher data indicating that the third observation data corresponds to the action requiring attention or the condition requiring attention;
The second learning data includes second teacher data indicating that the second observation data does not correspond to the action requiring attention or the condition requiring attention.
The learning method according to claim 9.