JP2007272361A - Plant control equipment - Google Patents

Abstract

To provide a plant control device in which a simulation model can be easily created.
A model 500 that predicts a value of a measurement signal 1 obtained from a control object 100 when a predetermined operation signal 16 is given to the control object 100 and a model output 13 that is a prediction result of the model 500 are modeled. A learning unit 600 that learns how to generate the model input 12 to be given to the model 500 so as to converge to the output target value, and an operation signal generation unit 300 that generates the operation signal 15 according to the result of the learning unit 600. The plant control apparatus in which the operation signal 15 generated by the generation unit 300 is used as the operation signal 16 includes an external input interface 210 that takes in the measurement signal 1 and a measurement signal database 230 that stores the value of the measurement signal 2. The learning unit 600 calculates the average and variance of the measurement signals stored in the measurement signal database 230, and Those to modify the operation signal 15 by the operation signal generating unit 300 using the result of the dispersion and.
[Selection] Figure 1

Description

  The present invention relates to a plant control device such as a power generation facility, and more particularly to a control device suitable for controlling boiler equipment.

  In the control of a plant such as a thermal power generation facility, usually, a measurement signal obtained from a plant to be controlled is processed, and an operation signal to be given to the control target is calculated. For this reason, the control device is implemented with an algorithm for calculating the operation signal so that the plant measurement signal taken from the control target achieves the operation target.

  As a control algorithm used for plant control at this time, a so-called PI (proportional / integral) control algorithm is conventionally known. Here, the PI algorithm is to derive the operation signal by multiplying the deviation between the operation target value and the measurement signal by a proportional gain, and adding a value obtained by time-integrating the deviation to the value. At this time, a plant operation signal may be derived using a learning algorithm.

  By the way, some parameters exist in the control algorithm used for plant control, but it is necessary to tune these parameters in advance to values suitable for the control target. In general, tuning of the parameter is performed on a control target that is simulated (simulated) using a physical model or a statistical model.

  Here, especially in the case of statistical models, the neurons of the human neural network are represented by elements called nodes that are simulated by linear or nonlinear functions, arranged in layers, and moved from the previous layer to the next layer. A method using a so-called neural network that artificially simulates a network structure through which a signal is transmitted is well known.

  The model using the neural network is a model in which a desired output signal is output by adjusting parameters in the model by providing an input signal and a desired output signal as a teacher signal. In this way, when a model simulating a control target is used as a model, an operation signal input to the control target may be used as an input signal to the model, and a measurement signal from the plant may be used as an output signal of the model.

  At this time, as a method for adjusting the basic structure of the neural network and the parameters in the model, for example, the Back Propagation method, or the back propagation through time which is a learning method of the neural network having a feedback mechanism, is used. (Back Propagation Through Time) method and the like (for example, see Non-Patent Document 1).

  On the other hand, unlike the case of learning by giving a teacher signal as in a neural network, a technique called reinforcement learning has been actively studied in the field of unsupervised learning. Here, this reinforcement learning is a framework of learning control that adapts to the environment through trial and error, and if you acquire the state of the environment and act on it, you will be rewarded according to its content As the reward at this time is the correct action with respect to the environment or the action that reaches the target of the environment, the more rewards can be obtained.

  Therefore, in this case, an action that obtains more reward is selected as a target, and as a result, the action that reaches the target aimed by the environment is adapted. At this time, an agent that selects an action for obtaining more rewards with respect to the environment is generally called an agent. If the environment is regarded as a control object and the agent as a controller, the control object and the trial and error are considered. The generation method of the operation signal given to the controlled object is learned so that the measurement signal obtained from the controlled object becomes desirable through the dynamic interaction, and this is known as a learning control framework. is there.

  In this reinforcement learning, it can be obtained from the current state to the future using the evaluation value of the scalar quantity calculated by using the signal obtained from the controlled object (this is called reward in reinforcement learning). The operation signal generation method that maximizes the expected value of the evaluation value is learned, but the operation signal generation method at this time is positive when the measurement signal reaches the driving target value. There is known a method of giving an evaluation value and learning using an algorithm such as Actor-Critic, Q-learning, or real-time dynamic programming (see, for example, Non-Patent Document 2).

  Further, a framework called Dyna-architecture is known as a method developed from the above-described method (see Non-Patent Document 1). In this framework, a model for simulating a control target is provided in the control device. In this case, the model takes an operation signal given to the controlled object as a model input, and calculates a model output that is a predicted value of the measured signal to be controlled. The model at this time is constructed using physical formulas and statistical methods.

Then, a model input generation method is learned by using the evaluation value calculated using this model output as a clue. In this Dyna-architecture, a model input generation method that achieves the model output target value is learned in advance. In addition, an operation signal to be applied to the controlled object is determined according to the learning result.
“Neural Network and Measurement Control” by Nishikawa Hoichi and Shinzo Kitamura, Asakura Shoten, published January 25, 1995 “Reinforcement Learning” by Sadayoshi Mikami and Masaaki Minagawa Morikita Publishing Co., Ltd. December 20, 2000

  As mentioned above, when designing a plant control device, it is necessary to create a model that appropriately simulates the controlled object. In order to improve, detailed physical models and numerical analysis are required. This numerical analysis requires creation of a mesh (calculation grid), and an increase in the number of meshes is necessary for improving accuracy.

  For example, when analyzing a combustion phenomenon for a large plant such as a boiler of a thermal power plant, a large number of meshes are required. For this reason, measures to shorten the calculation time by increasing the algorithm speed or parallel calculation have been used, but it is still difficult to calculate various operating conditions continuously. .

  In addition, when considering simulation of a control target using a statistical model, the data used to create the model is well fitted (fit), but the phenomenon that the accuracy when a different value is input is significantly reduced occurs. This phenomenon is generally referred to as over-learning, but the conventional technology requires a device to avoid this phenomenon and create a general-purpose model. There were restrictions.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to provide a plant control apparatus in which a simulation model can be easily created.

  The purpose is to generate an operation signal necessary for the measurement signal value obtained from the control object to be within the operation target value of the control object when a predetermined operation signal is given to the control object. A plant control apparatus that uses the operation signal as the predetermined operation signal, and predicts the value of the measurement signal obtained from the control object when the predetermined operation signal is given to the control object. And learning means for learning a method of generating a model input to be given to the model so that the model output which is a prediction result of the model converges to the model output target value, and given to the control object according to the result of the learning means In a plant control apparatus that includes an operation signal generation unit that generates an operation signal, and the operation signal generated by the operation signal generation unit is the predetermined operation signal. An external input interface that captures the measurement signal to be controlled and a measurement signal database that stores the value of the measurement signal captured by the interface, and calculates the average and variance of the measurement signal stored in the measurement signal database. This is achieved by modifying the operation signal using the result of averaging and variance and newly generating the predetermined operation signal.

  Further, the above-described object is that when a predetermined operation signal is given to the control object, an operation signal necessary for the value of the measurement signal obtained from the control object to fall within the operation target value of the control object. Is generated and the operation signal is used as the predetermined operation signal, and when the predetermined operation signal is given to the control target, the value of the measurement signal obtained from the control target is predicted. And a learning means for learning a generation method of a model input given to the model so that a model output that is a prediction result of the model converges to a model output target value, and the control target according to a result of the learning means The plant control device has an operation signal generation means for generating an operation signal to be applied to the plant, and the operation signal generated by the operation signal generation means is used as the predetermined operation signal. An external input interface for capturing the measurement signal to be controlled, and a measurement signal database for storing the value of the measurement signal captured by the interface, and calculating an average and variance of the measurement signals stored in the measurement signal database. When the operation signal is modified to generate a new operation signal, the change width of the operation signal is determined based on the variance of the measurement signal.

  At this time, a function for generating an operation signal using a result of calculating an expected value from the average and variance of the measurement signal may be provided, and as an external input function, a distribution shape of the measurement signal is input to the control device. In addition, as an external input function, a user interface for inputting at least one of the average value, expected value, variance, and distribution shape of the measurement signal to the control device may be provided. May be.

  Further, at this time, the control target is a thermal power plant, and among the measurement signals of the thermal power plant, a function of taking at least one of carbon monoxide and nitrogen oxide into the control device, and an external input function, Even if it has a function of setting at least one environmental regulation value of carbon monoxide and nitrogen oxide as a limit value of the measurement signal and a function of generating an operation signal of at least the air damper opening according to the learning result Furthermore, among the measurement signals of the thermal power plant, a function of taking at least one of carbon monoxide and nitrogen oxide into the control device, and an external input function are carbon monoxide, nitrogen oxide, and carbon dioxide. , Sulfur oxide, mercury, fluorine, fine particles of dust or mist, and at least one environmental regulation value of volatile organic compounds According to the learning result and the function to be set as at least one of air damper opening, fuel flow rate supplied to the burner, burner air flow rate, air flow rate supplied to the air port, gas recirculation amount, burner angle, supply air temperature The above object can also be achieved by providing a function for generating a signal.

  According to the present invention, the average and variance of the measurement signal are calculated, and a model for simulating the controlled object is created from the calculation result. Therefore, the model for simulating the controlled object corresponds to the accumulation of data. The distribution shape is incorporated, and fluctuations in data can be known from the size of the variance.

  As a result, it can be seen that when the variance is large, the influence of the plant operating state or other process values is large, and when the variance is small, the influence of the plant operating state or other process values is small. Therefore, according to the present invention, by constructing a control algorithm in consideration of the magnitude of dispersion, it is possible to avoid low reliability due to data fluctuations and a small amount of accumulated data.

  Hereinafter, a plant control apparatus according to the present invention will be described in detail with reference to embodiments shown in the drawings. FIG. 1 shows an embodiment in which a plant control apparatus according to the present invention is applied to a control object 100. For this purpose, a control apparatus 200, an input apparatus 900, a maintenance tool 910, and an image display apparatus 950 are provided. .

  First, the control device 200 captures the measurement signal 1 from the control target 100 via the external input interface 210, and transmits the operation signal 16 to the control target 100 via the external output interface 220. The measurement signal 2 captured by the external input interface 210 is transmitted to the operation signal generation unit 300 and is stored in the measurement signal database 230 together with this. The operation signal 15 generated by the operation signal generation unit 300 is transmitted to the external output interface 220 and is stored in the operation signal database 240 together with this.

  The operation signal generation unit 300 generates the operation signal 15 using the information stored in the control logic database 250 and the learning information database 280 so that the measurement signal 1 from the control target 100 achieves the driving target value. At this time, the information stored in the learning information database 280 is generated by the learning unit 600. For this reason, the learning unit 600 is connected to the model 500.

  Here, the model 500 has a function of simulating the characteristics of the controlled object 100. That is, the model input 12 for operating the model 500 is given to the model 500 in the same manner as when the operation signal 16 is given to the control object 100 and the measurement signal 1 is obtained, and as a result, the model output 13 is obtained. is there. At this time, the model output 13 is a predicted value of the measurement signal 1. Therefore, the model 500 simulates the characteristics of the controlled object 100 and has a function of calculating the model output 13 for the model input 12 using a model formula based on a physical law or a statistical method.

  The model creation unit 400 has a function of generating a model 500 from the previous model parameter 5 and the measurement signal 3 stored in the model parameter database 270. The model creation unit 400 also has a function of newly generating a model 500 using the model parameter generated by a random number or the like and the measurement signal 3 when the model parameter database 270 does not have the model parameter 5 last time. have.

  Therefore, the learning unit 600 generates a model input 12 using the previous learning information 11 stored in the learning information database 280, the learning parameter 7 stored in the learning parameter database 260, and the model output 13. For this reason, the evaluation value 14 calculated using the model output 13 calculated by the model 500 is input to the learning unit 600. In the learning unit 600, the learning information is updated using the evaluation value 14, and the updated learning information 10 is transmitted to the learning information database 280.

  The operation signal generation unit 300 generates the operation signal 15 using the learning information 9 stored in the learning information database 280 and the control logic information 6 stored in the control logic database 250. At this time, the operator of the control target 100 is provided in the control device 200 by using the input device 900 configured by the keyboard 901 and the mouse 902 and the maintenance tool 910 connected to the image display device 950. You can access information stored in various databases.

  The maintenance tool 910 includes an external input interface 920, a data transmission / reception processing unit 930, and an external output interface 940. An input signal 31 generated by the input device 900 is sent to the maintenance tool 910 via the external input interface 920. It is captured. At this time, the data transmission / reception processing unit 930 acquires the database information 30 provided in the control device 200 according to the information of the input signal 32.

  The data transmission / reception processing unit 930 transmits an output signal 33 obtained as a result of processing the database information 30 to the external output interface 940. The output signal 34 is supplied from the external output interface 940 to the image display device 950 and displayed as an image in preparation for the operator's monitor.

  In this embodiment, all necessary databases are arranged inside the control device 200, but these can also be arranged outside the control device 200. Further, in this embodiment, all the signal processing functions for generating the operation signal 16 are arranged inside the control device 200, but these may be arranged outside the control device 200.

  Next, the operation of this embodiment will be described below along with the information stored in the database and the signal processing function, taking the case where the present invention is applied to a thermal power plant as an example. Here, a thermal power plant that is to be controlled 100 will be described with reference to FIG. Here, the case where coal is used as fuel will be described. In this case, coal is stored in the coal bunker 111. The coal bunker 111 supplies coal to the mill 110 via the coal feeder 112.

  In the mill 110, the coal is finely crushed by an internal roller to be pulverized coal, so-called pulverized coal. And this pulverized coal is conveyed with the primary air for coal conveyance to the burner 102 with the secondary air for combustion adjustment, is supplied in the furnace of the boiler 101, and is combusted. At this time, the primary air is supplied to the mill 110 via the pipe 133, the pulverized coal and the primary air are guided to the burner 102 via the pipe 134, and the secondary air is guided to the burner 102 via the pipe 141. It is burned.

  At this time, the after-air for two-stage combustion is supplied into the furnace of the boiler 101 via the after-air port 103, and this after-air is guided via the pipe 142. The high-temperature gas generated in the furnace due to the combustion of coal flows along the predetermined path including the heat exchanger 106 of the boiler body in the furnace of the boiler 101, and then passes through the air heater 104 to treat the exhaust gas. And then released to the atmosphere through the chimney.

  At this time, the feed water circulating through the heat exchanger 106 of the boiler 101 is pressurized by the feed water pump 105 and introduced into the boiler 101 and heated by the heat exchanger 106 to become high-temperature and high-pressure steam. In this example, one heat exchanger is provided, but a plurality of heat exchangers may be arranged.

  The steam that has passed through the heat exchanger 106 and became high temperature and pressure is guided to the steam turbine 108 through the turbine governor 107, where the energy of the steam is converted into rotational energy, and the generator 109 is rotationally driven. As a result, electric power is generated. At this time, the exhaust gas from the steam turbine 108 is sent to the condenser 113, and as a result of cooling here, it becomes condensed water and is sent to the feed water pump 105 again. In this process, air is extracted from the turbine 108, and a device for heating the feed water by the extracted steam is arranged to improve the thermal efficiency.

  By the way, various measuring instruments are arranged in such a thermal power plant. For example, FIG. 2 illustrates a flow rate measuring device 150, a temperature measuring device 151, a pressure measuring device 152, a power generation output measuring device 153, and a concentration measuring device 154. And in the flow measuring device 150, the flow volume of the water supply supplied to the boiler 101 from the water supply pump 105 is measured. The temperature measuring device 151 and the pressure measuring device 152 measure the temperature and pressure of the steam supplied to the steam turbine 108. The amount of power generated by the power generator 109 is measured by a power generation output measuring device 153.

  On the other hand, CO (carbon monoxide), NOx (nitrogen oxide), carbon dioxide, sulfur oxide, mercury, fluorine, dust or mist fine particles contained in the gas passing through the boiler 101, volatile organic Information on the concentration of the component such as at least one environmental regulation value of the compound is measured by the concentration measuring device 154. In general, many measuring instruments other than those shown in FIG. 2 are arranged in the thermal power plant, but are omitted in FIG. Information acquired from these measuring instruments is shown as measurement information 1 output from the control object 100 in FIG. 1, and these are transmitted to the control device 200.

  Next, the paths of the primary air and secondary air supplied from the burner 102 and the after-air supplied from the after-air port 103 will be described. First, the primary air is taken into the pipe 130 from the fan 120, and is branched into a pipe 132 that passes through the air heater 104 and a pipe 131 that does not pass along the way, and then joins the pipe 133 and is guided to the mill 110. At this time, the air passing through the air heater 104 is heated by the gas and used to convey the pulverized coal produced by the mill 110 to the burner 102.

  On the other hand, the secondary air and the after air are taken into the pipe 140 by the fan 121 and heated by the air heater 104, and then branched into the secondary air pipe 141 and the after air pipe 142, respectively, and the burner 102 and the after air, respectively. It is led to the airport 103.

  FIG. 3 is an enlarged view of the air heater 104 and the piping section through which the primary air and secondary air and after-air pass at this time, and as shown in this figure, each pipe has an air damper 160, 161, 162, and 163 are arranged, and by operating these air dampers, the area through which air passes through the pipe can be changed, and the flow rate of air passing through the pipe can be adjusted by operating the air damper. Therefore, the control device 200 operates devices such as the water supply pump 105, the mill 110, and the air dampers 160, 161, 162, and 163 using the operation signal 16 generated there.

  Next, information stored in the measurement signal database 230 and the operation signal database 240 will be described with reference to FIGS. 4 is an example of information stored in the measurement signal database 230, and FIG. 5 is an example of information stored in the operation signal database 240.

  First, in the measurement signal database 230, as shown in FIG. 4, information measured on the control target 100 is stored together with each measurement time for each measuring instrument. For example, the flow rate value F measured by the flow meter 150 in FIG. 2, the temperature value T measured by the temperature meter 151, the pressure value p measured by the pressure meter 152, the pressure value p measured by the power generation output meter 153, The power generation output value E and the NOx concentration D contained in the exhaust gas are stored together with time information.

  At this time, in order to make it easy to use the data stored in the measurement signal database 230, a unique number called a PID number is assigned to each measurement value as shown in the figure. In FIG. 4, data is stored at a cycle of 1 second, but the cycle at this time, that is, the sampling cycle of data collection can be arbitrarily set.

  Next, as shown in FIG. 5, the operation signal database 240 stores operation signals such as a feed water flow rate command signal together with time information. Here, it goes without saying that a unique PID number is assigned to each operation signal, and the time interval can be arbitrarily set.

  Next, operations of the model creation unit 400 and the model 500 will be described. The model 500 realizes the relationship of the measurement signals shown in FIG. 6 by the structure shown in FIG. Here, FIG. 6 is a plot of the relationship between the air flow rate ratio and the measurement signal A. At this time, the number of data that can be plotted on the graph varies depending on the state of the plant. For example, in a new plant, since it is obtained from design value information, the number of data is small. On the other hand, the number of data increases in a plant with many years of operation.

  Thus, since the number of data varies depending on the plant situation, here, a distribution is assumed for each data, and the difference in the number of data is expressed by the shape of the distribution. Then, when the number of data is small, the variance is large, so the distribution is widened. On the other hand, when the number of data is large, the variance is small and the distribution is sharp. At this time, if there is prior information on the data, the distribution shape can be assumed. However, in the case of new data, it is necessary to estimate the distribution based on the data obtained without the prior information.

  There are many known methods for estimating the distribution from only the data, but in any case, the normal distribution is based on the central limit theorem that the distribution approaches the normal distribution by increasing the number of data, regardless of the population distribution. Can be assumed. If the distribution can be assumed, the shape can be determined from the mean and the variance. The assumption of normal distribution based on the central limit theorem is described in detail in, for example, “Introduction to Statistics”, the Department of Statistics, Faculty of Liberal Arts, The University of Tokyo, Tokyo University Press, published on July 10, 1991.

  FIG. 7 is a diagram for explaining a model structure when the distribution is determined. As an output signal at this time, a model that outputs the median value and the variance value of the distribution shown in FIG. It consists of an intermediate layer and an output layer, and has a structure in which nodes of each layer are mutually coupled. The node portion uses a linear or nonlinear function, but generally uses a sigmoid function. Each node connection has a weighting coefficient, and represents the strength of the mutual relationship between the nodes.

  Usually, the model parameter (described later) refers to this weighting factor. Here, the intermediate layer is expressed as a single layer, but may be expressed as a multilayer. A related measurement signal is input to the input signal. By simulating the controlled object with this model, it is possible to create a model that takes into account the number of stored data, and thus various states of the controlled object can be easily simulated.

  FIG. 8 is a flowchart showing a process for creating the model 500 by the model creation unit 400. The parameters necessary for executing this flowchart are stored in the model parameter database 270. The form of information stored in this database will be described later.

  When the processing according to the flowchart of FIG. 8 is started, first, in step 401, it is selected whether to use a model parameter set in the past or create a new model parameter. When a new model parameter is created here, the process proceeds to step 402, and the initial value of the model parameter is set using a random number.

  Next, in step 403, the measurement signal 3 that is the input signal and output signal of the model 500 is extracted from the measurement signal database 230, and the average of the measurement signal 3 that is the output signal of the model 500 is calculated. The calculated average is stored in the learning information database 280.

  In step 404, the variance of the measurement signal 3 that is the output signal of the model 500 is calculated. Here, when there is only one sample of the measurement signal, the variance cannot be calculated. Therefore, in this case, a larger dispersion value is given as a default value. For example, 100 or the like may be set as a default value. This default value can be sequentially changed by the user.

  As the shape of the distribution at this time, the shape stored in the learning information database 280 is used. However, when the shape is not yet stored in the learning information database 280, a normal distribution is used. The variance thus calculated is stored in the learning information database 280.

  In step 405, the mean and variance calculated in steps 403 and 404 are set as the teacher signal of the model 500, and then in step 406, parameters necessary for learning such as the number of learning, the learning coefficient, and the number of nodes are set. When a new model parameter is created, a default value stored in the model parameter database 270 is used.

  In step 407, the model parameters are sequentially updated from the initial values by learning. The Back Propagation method or the like is used as a model parameter update method by learning. This learning method is described in detail in “Neural network and measurement control” written by Junichi Nishikawa and Shinzo Kitamura, Asakura Shoten, published on January 25, 1995. The model parameters are updated so that there is no difference between the output signal and the teacher signal when the signal is given.

  Here, the difference between the output signal from the model 500 and the teacher signal is generally expressed by a square error and is called an evaluation function. A variation of the evaluation function when each model parameter is varied is subjected to partial differential calculation, and a value obtained by multiplying the obtained value by the learning coefficient is used as an update of the model parameter. Therefore, if this is repeated, the difference between the output signal of the model 500 and the teacher signal disappears, and the evaluation function approaches zero.

  When the evaluation function approaches zero, the partial differential value also approaches zero, and the update amount of the model parameter approaches zero. However, in the numerical calculation, since it is never completely zero, if the evaluation function falls below the set value in step 408, it is considered that the learning is finished, and the model creation is finished.

  On the other hand, if the learning end condition is not satisfied in step 408, the iterative calculation is stopped when the number of learning repetitions reaches the set number, and the process returns to step 406 to set the learning parameters again.

  Returning to step 401, when the use of the past model parameter is selected here, it is selected in step 409 whether or not the past model parameter is to be corrected by learning using the past model parameter as an initial value. In the case of correction, the process proceeds to step 403. If not corrected, since the past model parameters are used as they are, it is not necessary to reconstruct the model 500, and the model creation ends.

  In this embodiment, a neural network using a sigmoid function is used for a node, but other network models such as a radial basis function network using a Gaussian function for a node may be used.

FIG. 9 is a diagram for explaining the form of information stored in the model parameter database 270. As shown in FIG. 9, the model parameter database 270 includes an ID, a creation date, a learning coefficient, the number of learnings, an end condition, The number of nodes and parameter values are saved. Here, the number of nodes is divided into an input layer, an intermediate layer, and an output layer. Further, the parameter value is a weighting factor, and there are mutual connections of nodes, which are stored as W ₁₁ , W ₁₂ ,.

  An ID value of 000 indicates a default value of a learning parameter when a model parameter is newly created. For new creation, the number of nodes and parameter values are usually blank.

  Next, operations of the model 500 and the learning unit 600 will be described. The learning unit 600 learns the generation method of the model input 12 so that the model output 13 achieves the model output target value for the model 500 that simulates the characteristics of the controlled object 100. Examples of algorithms for performing such learning are described in “Reinforcement Learning”, co-translation by Sadayoshi Mikami and Masaaki Minagawa, Morikita Publishing Co., Ltd., published on December 20, 2000. There is reinforcement learning theory.

  Here, the reinforcement learning is to learn a generation method of the model input 12 for achieving the model output target value through the interaction between the learning unit 600 and the model 500 using the evaluation value (reward) information as a clue. By applying this reinforcement learning, it is possible to learn the generation method of the model input 12 that maximizes the expected value of the evaluation value obtained from the current time to the future.

  In this embodiment, a case where the Q-1earning method is applied will be described as an example of the reinforcement learning algorithm. However, as a learning method in the control device 200 of this embodiment, a genetic algorithm is used in addition to the reinforcement learning method. It is also possible to apply optimization techniques such as linear and nonlinear programming.

  FIG. 10 is a schematic diagram of the Q-1earning method. As shown here, the learning unit 600 to which the Q-1earning method is applied includes an agent 650 that generates the model input 12 and an evaluator that evaluates the value of the state. 660.

  11 and 12 are flowcharts for explaining processing in the case of the Q-1earning method. Here, the learning parameters database 260 and the learning information database 280 are stored for the design parameters necessary for executing this flowchart, such as the discount rate γ. The form of information stored in these databases and a method for registering design parameters in the database will be described later.

  In FIG. 11, first, this flowchart is repeatedly performed while the control target 100 is being controlled. In the first step 301, the sampling period r in the control is acquired. Next, in step 302, one episode learning is executed. In step 302, the model 500 and the learning unit 600 operate to execute the above-described reinforcement learning algorithm. In step 303, a learning end determination is executed.

  This step 303 is a step provided for ending the learning within the control sampling period or less, returning to step 302 while the learning execution time is smaller than r, and ending learning when the processing time exceeds the period r. .

  FIG. 12 is a flowchart for explaining the operation at the time of executing one episode learning in step 302 of FIG. 11. First, in step 601, initial values of model inputs are set at random. Next, in step 602, the model input 12 generated in step 601 is input to the model 500, and the model output 13 is obtained. Next, in step 603, the model output 13 is compared with the target value of the model output. If the model output 13 has achieved the model output target value, the episode is terminated, and if not, the process proceeds to step 604.

  In the next step 604, the learning unit 600 determines the model input change width Δa using information stored in the learning information database 280. A method for determining the model input change width Δa will be described later.

In step 605, the model input 12 is determined using the following equation (1).

In step 606, the model input 12 determined in step 605 is input to the model 500, and the model output 13 is obtained. Next, at step 607, an evaluation value is calculated by the following equation (2) based on the model output 13 obtained at step 606.

Here, the value Q (s, a) is determined by the sum of time, but this has meaning. That is, an actual action, in this case, a model input 12 is generated and a response is obtained when it is input to the model 500, but this often involves a delay time. In particular, this effect is significant when applied to a plant.

  Therefore, it is more realistic to determine the value based on the sum of rewards given in the future, instead of determining the value based on the reward immediately after the action, so it is determined by the sum at the time. is there. Also, in this case, by introducing the discount rate γ, there is an advantage that an evaluation value considering responsiveness can be obtained by setting the reward obtained immediately after the action to be high.

In step 608, based on the evaluation value calculated in step 607, the parameter of the agent is updated by the following equation (3), and the updated result is stored in the learning information database 280.

Finally, in step 609, the end determination is performed by the same method as in step 403. That is, in step 609, if the learning end condition is not satisfied, the iterative calculation is stopped when the number of times of learning reaches the set number of times, and the process returns to step 604.

  Next, processing when the model input 12 is generated in the agent 650 of the learning unit 600 and the state value is calculated in the evaluator 660 will be described. Although the case where the tar coding method is used will be described here, the agent 650 and the evaluator 660 may be configured using a method other than this method.

  First, as described above, the evaluator 660 divides the state by the tile coding method. The tile coding method is a method of recognizing a continuous state as a discrete state by dividing an input space and determining which region it belongs to. FIG. 13 shows a tile coding method at this time. In this figure, each region is called a tile. For example, when the input signal 12 to the model 500 is a two-dimensional input signal A and an input signal B, the input signal A is between 0 and 1, and the input signal B is between 1 and 2, It belongs to the tile of state number 1.

  In this case, information is stored in the learning information database 280 in a form in which the state number and the value function correspond as shown in FIG. The evaluator 660 uses the value of the input signal 12 when the model output 13 is obtained and the information stored in the learning information database 280 to calculate the value of the state according to the above-described equation (3).

  First, FIG. 15 shows information stored in the learning information database 280. As shown in FIG. 15, the average of the teacher signals used when creating the model 500 corresponding to the state number is shown. The variance is preserved. At this time, in step 604 described above, the model input change width Δa is determined based on the variance value of the teacher signal.

  Therefore, when the variance is small, the input change width Δa is increased because the variation is small and the sensitivity to changes in the input signal is low. On the other hand, when the variance is large, the input variation width Δa is reduced because the variation is large and the sensitivity to changes in the input signal is high.

  Next, FIG. 16 shows an aspect of information stored in the learning parameter database 260. This is performed by executing steps 606 and 607 in the flowchart of FIG. 12 as shown in FIG. Parameters such as the required learning rate are saved. In this reinforcement learning, since the generation method of the model input 12 is learned so that the expected value of the evaluation value is maximized, the evaluation value increases when the model output 13 achieves the model output target value. Is desirable.

  Therefore, as a method of generating such an evaluation value, there is a method in which a positive value, for example, “1” is used as the evaluation value when the model output 13 achieves the model output target value. Moreover, when the model output target value is not achieved, there is a method of calculating the evaluation value using a function that is inversely proportional to the error between the model force target value and the model output 13. Furthermore, a method of calculating an evaluation value by combining these methods is also conceivable.

  Next, a method in which the operator of the control target 100 displays the database information on the image display device 950 using the maintenance tool 910 will be described with reference to FIGS. In this case, the operator uses the keyboard 901 and the mouse 902 to execute an operation such as inputting a parameter value in a blank area of the displayed screen.

  Here, FIG. 17 shows an initial screen displayed on the image display device 950. Here, the operator selects a necessary button from among the control logic creation / edit button 951, the learning condition setting button 952, and the information display button 953. By selecting, moving the cursor 954 using the mouse 902, and clicking the mouse 902, one of the buttons is pressed.

  First, FIG. 18 is a control logic editing screen displayed when the control logic creation / editing button 951 is clicked. In this screen, the operator clicks either the new creation button 967 or the editing button 968. Push. If it is newly created, a logic diagram in which nothing is described is opened. In the case of editing, the logic to be edited is selected and the logic diagram is displayed. At the time of creation or editing, a necessary module is selected from the standard element modules 963 registered in advance and moved to the logic editing screen 961. The modules are connected using a connection / erase 962.

  The control logic drawing created on the display screen of FIG. 18 is saved in the control logic database 250 via the data transmission / reception processing unit 930 when the save button 964 is clicked. Further, the operation signal generation unit 300 generates the operation signal 15 when the measurement signal 2 is input, using the information in the control logic drawing. Further, the operation signal generation unit 300 can generate the operation signal 15 by using information stored in the learning information database 280 together.

  At this time, the learning information database 280 stores the state number and central information of the information shown in FIG. Therefore, by using these pieces of information, it is possible to easily generate the operation signal 15 having the same value as the model input 12 so that the model output 13 becomes a desired value. If the control logic drawing created at this time is not saved, a cancel button 965 is clicked. On the other hand, by clicking the return button 966, it is possible to return to the screen of FIG.

  When the learning condition setting button 952 is clicked on the screen of FIG. 17, the screen of FIG. 19 is displayed. Therefore, the operator inputs the learning coefficient, the number of times of learning, and the end condition necessary for executing the flowchart of FIG. 8 to the model creation field 971 in the screen of FIG. 19 based on the model-specific PID, Alternatively, if already entered, the value can be corrected. At this time, the operator can change the default value whose ID is 000.

  Next, in the parameter setting field 972, setting parameters necessary for executing the flowcharts of FIGS. 11 and 12 are input. Further, in the operation end setting column 973, the operation end name for learning the operation method, the operation range, and the division number for tile coding are input according to the flowchart of FIG. Here, when the next page button 977 in FIG. 19 is clicked, the second half screen of the learning condition setting screen is displayed. The previous page button 978 and the second half screen of the learning condition setting screen will be described later.

  Then, by clicking the save button 974 in FIG. 19, the information input in the model creation field 971 is stored in the model parameter database 270, the information input in the parameter setting field 972 is stored in the learning parameter database 260, and the operation end setting is performed. Information input in the column 973 is stored in the learning information database 280, respectively.

  Here, when the cancel button 975 is clicked, all the information input to the model creation field 971, the parameter setting field 972, and the operation end setting field 973 are canceled. Then, a return button 976 is clicked to return to the screen of FIG.

  Next, the first half screen of the learning condition setting screen will be described with reference to FIG. This first half screen is displayed by clicking the next page button 977 in FIG. Therefore, the operator can input the average, variance, and distribution shape of the output signal of the model 500 in the learning information field 979, or can correct them if they are input. Based on this information, the model input change width at step 604 in the flowchart of FIG. 12 is determined.

  Next, FIG. 21 is a screen used when setting the conditions for displaying information stored in the measurement signal database 230 and the operation signal database 240 on the image display device 950. In FIG. It is displayed by clicking 953. Therefore, the operator inputs a measurement signal or an operation signal to be displayed on the image display device 950 into the input field 981 together with the range (upper limit / lower limit). At this time, the time to be displayed is input in the time input field 982.

  Also, by clicking the display button 983, a trend graph as shown in FIG. 22 is displayed on the image display device 950. By clicking the return button 991 here, it is possible to return to the screen of FIG. On the other hand, by clicking the return button 984, it is possible to return to the screen of FIG. In addition to the images described above, information stored in the database in the control device 200 can be arbitrarily selected and displayed on the image display device 950 in any manner.

  Next, in this embodiment, the control target 100 in FIG. 1 is the thermal power plant described with reference to FIG. 2, and the control device 200 is applied thereto, and by operating the air damper of the thermal power plant, CO 2 It is possible to control the emission status of at least one environmental regulation value such as NOx, carbon dioxide, sulfur oxide, mercury, fluorine, fine particles composed of dust or mist, and volatile organic compounds.

  Here, first, FIG. 23 explains the basic characteristics of CO and NOx discharged in a thermal power generation facility. Generally, the amount of CO and the amount of NOx are in a trade-off relationship as shown in the figure, and CO is When trying to reduce, NOx tends to increase, and when trying to reduce NOx, CO tends to increase.

  On the other hand, in a thermal power plant, the amount of CO and NOx emitted from the chimney is legally restricted, especially for NOx, and therefore the gas at the boiler outlet is led to the denitration equipment, where The amount of consumption of ammonia used in the denitration device at this time increases as the NOx concentration at the inlet of the denitration device increases.

  Therefore, reducing the amount of NOx at the inlet of the denitration apparatus is a great cost merit. Therefore, it is desirable to reduce the NOx concentration as much as possible. Therefore, a control algorithm considering the trade-off relationship between CO and NOx is required. Become. Moreover, when the thermal power plant differs in design, trial operation, and operation, the stored measurement signal data will also differ. Therefore, if the operating conditions are different even in long-term operation, just because the number of accumulated data is large, it is not always preferable.

  However, in the case of the above embodiment, the model simulating the thermal power plant to be controlled can model the distribution shape according to the accumulated data according to the flowchart of FIG. From this, the past state can be grasped. In other words, if the variance is large, it means that there is a lot of variation, and it can be seen that the state of the plant is very unstable. If the variance is small, the variation is small, so the state of the plant is very stable. Therefore, it is possible to construct a control algorithm considering the reliability of accumulated data from the flowchart of FIG.

  As a result, according to the above embodiment, even when the accumulated data is small, it is possible to construct a control algorithm that takes into consideration that the number of data is small. In consideration of the trade-off relationship between CO and NOx, the legal regulations for these can always be satisfied.

  In addition to the above-mentioned CO and NOx, there are carbon dioxide, sulfur oxide, mercury, fluorine, dust, mist, volatile organic compounds, etc. in addition to the above-mentioned CO and NOx. As described above, according to the embodiment, it is possible to control the at least one environmental regulation value.

It is a block diagram which shows one Embodiment of the control apparatus of the plant which concerns on this invention. It is a block diagram which shows an example of the thermal power plant used as control object in one Embodiment of this invention. It is an enlarged view of a piping part and an air heater part in an example of a thermal power plant to be controlled in one embodiment of the present invention. It is explanatory drawing which shows the aspect of the data memorize | stored in a measurement signal database in one Embodiment of this invention. It is explanatory drawing which shows the aspect of the data memorize | stored in the operation signal database in one Embodiment of this invention. It is explanatory drawing which shows the mechanism of modeling used in one Embodiment of this invention. It is explanatory drawing of the modeling structure used in one Embodiment of this invention. It is a flowchart figure for demonstrating the process of the model preparation part in one Embodiment of this invention. It is explanatory drawing which shows the aspect of the data memorize | stored in a model parameter database in one Embodiment of this invention. It is a block diagram of the learning part by the Q-Learning method used in one Embodiment of this invention. It is a flowchart figure of the algorithm currently used for the learning part by one Embodiment of this invention. It is a flowchart figure at the time of 1 episode learning execution in the algorithm currently used for the learning part by one Embodiment of this invention. It is explanatory drawing of the tile coding applied to the evaluator in the learning part by one Embodiment of this invention. It is explanatory drawing which shows the aspect of the data memorize | stored in a learning information database in one Embodiment of this invention. It is explanatory drawing which shows the aspect of the data memorize | stored in a learning information database in one Embodiment of this invention. It is explanatory drawing which shows the aspect of the data memorize | stored in the learning parameter database in one Embodiment of this invention. It is explanatory drawing of the initial screen displayed as an image in one Embodiment of this invention. It is explanatory drawing of the control logic creation / edit screen displayed as an image in one Embodiment of this invention. It is explanatory drawing of the first half screen of the learning condition setting screen displayed as an image in one Embodiment of this invention. It is explanatory drawing of the second half screen of the learning condition setting screen displayed as an image in one Embodiment of this invention. It is explanatory drawing of the display information setting screen displayed as an image in one Embodiment of this invention. It is explanatory drawing of the trend graph of the measured value displayed on an image in one Embodiment of this invention. It is a characteristic view explaining the relationship between CO and NOx discharged from a thermal power plant.

Explanation of symbols

100: Control target 200: Control device 210: External input interface 220: External output interface 230: Measurement signal database 240: Operation signal database 250: Control logic database 260: Learning parameter database 270: Model parameter database 280: Learning information database 300: Operation signal generation unit 400: Model creation unit 500: Model 600: Learning unit 900: Input device 901: Keyboard 902: Mouse 910: Maintenance tool 920: External input interface 930: Data transmission / reception processing unit 940: External output interface 950: Image display apparatus

Claims (7)

When a predetermined operation signal is given to the control target, an operation signal necessary to make the value of the measurement signal obtained from the control target fall within the operation target value of the control target is generated. A plant control apparatus in which a signal is the predetermined operation signal,
When a predetermined operation signal is given to the controlled object, the model that predicts the value of the measurement signal obtained from the controlled object and the model output that is the prediction result of the model converges to the model output target value. An operation signal generated by the operation signal generation unit includes a learning unit that learns a generation method of a model input to be given to the model, and an operation signal generation unit that generates an operation signal to be given to the control target according to a result of the learning unit. In the plant control apparatus in which the predetermined operation signal is used,
An external input interface for capturing the measurement target measurement signal;
A measurement signal database for storing measurement signal values captured by the interface;
A plant that calculates an average and variance of measurement signals stored in the measurement signal database, corrects the operation signal using a result of the average and variance, and newly generates the predetermined operation signal Control device. When a predetermined operation signal is given to the control target, an operation signal necessary to make the value of the measurement signal obtained from the control target fall within the operation target value of the control target is generated. A plant control apparatus in which a signal is the predetermined operation signal,
When a predetermined operation signal is given to the controlled object, the model that predicts the value of the measurement signal obtained from the controlled object and the model output that is the prediction result of the model converges to the model output target value. An operation signal generated by the operation signal generation unit includes a learning unit that learns a generation method of a model input to be given to the model, and an operation signal generation unit that generates an operation signal to be given to the control target according to a result of the learning unit. In the plant control apparatus in which the predetermined operation signal is used,
An external input interface for capturing the measurement target measurement signal;
A measurement signal database for storing measurement signal values captured by the interface;
When calculating the average and variance of the measurement signals stored in the measurement signal database and correcting the operation signal to generate a new operation signal, the change width of the operation signal is based on the variance of the measurement signal. A plant control device characterized by deciding. In the plant control apparatus according to claim 1 or 2,
A plant control apparatus comprising a function for generating an operation signal using a result of calculating an expected value from an average and variance of measurement signals. In the plant control apparatus according to claim 1 or 2,
A plant control device comprising a user interface for inputting a distribution shape of a measurement signal to the control device as an external input function. In the plant control apparatus according to claim 1 or 2,
A plant control device comprising a user interface for inputting at least one of an average value, an expected value, a variance, and a distribution shape of a measurement signal to the control device as an external input function. In the plant control apparatus according to any one of claims 1 to 5,
The controlled object is a thermal power plant,
Among the measurement signals of the thermal power plant, a function of capturing at least one of carbon monoxide and nitrogen oxide in the control device;
As an external input function, a function to set at least one environmental regulation value of carbon monoxide and nitrogen oxide as a limit value of the measurement signal,
A plant control device comprising a function of generating at least an operation signal of an air damper opening according to a learning result. In the plant control apparatus according to any one of claims 1 to 5,
The controlled object is a thermal power plant,
Among the measurement signals of the thermal power plant, a function of capturing at least one of carbon monoxide and nitrogen oxide in the control device;
As an external input function, at least one environmental regulation value of carbon monoxide, nitrogen oxide, carbon dioxide, sulfur oxide, mercury, fluorine, dust or mist, or volatile organic compound is set as the limit value of the measurement signal. With the ability to set as
According to the learning result, the function of generating at least one operation signal among the air damper opening, the fuel flow rate supplied to the burner, the burner air flow rate, the air flow rate supplied to the air port, the gas recirculation amount, the burner angle, and the supply air temperature. A plant control apparatus characterized by comprising:

Images