DE4230419A1 - Neural network with rule-base network configuration - Google Patents

Abstract

The neural network is pre-structural in a given network configuration using rule-base knowledge. Each initial value is defined by a normalised linear summation function in terms of weighted and unweighted base function. The latter is obtained from the mean values of localised positive function of the neural network input values.Pref. the base function is obtained via an exponential function of an inverse covariance matrix, multiplied by a normalisation factor for the base function.

Description

Information processing systems should be able to use knowledge in the form of rules to also process knowledge gained through experience. Here is a method that uses rule-based knowledge to pre-structure a neural network. As a result, the network already achieves good results in accordance with what has already been introduced before learning symbolic knowledge. With a small number of training data, this is achieved Network with this method good results, since areas of the entrance area for which none Training data are available to be covered by rule-based knowledge. To The training can be extracted and modified by an expert be interpreted.

The considered networks describe a map f: R ⁿ → R, with y = f (x). In the broadest sense, we consider a rule as domain-specific knowledge about the same input / output relationship. The rules are expressed in simple terms: if (premise) then (conclusion). The premise makes a statement about the entrance and the conclusion about the exit space.

{Rule _i | i: 1. . . M} is a set of rules. For each rule, a basic function b _i (x) is introduced, which is 1, where the rule applies and otherwise has the value 0. Alternatively, a value between 0 and 1 can be defined, which indicates the validity to which extent a certain rule applies to a given input vector. Such a basic function can e.g. B. can be described with a multivariable Gaussian function:

assuming that the matrix Σ _i , in which σ _ÿ ² is the jth diagonal element in Σ _i, is diagonal. The weighting factor _i is set to 1 here. The parameter A _j ⁱ = c _ÿ specifies the position of the jth dimension of the entrance space at which rule i is most valid. The parameter range _ÿ = σ _ÿ roughly specifies the range in which rule _i is valid in the jth input dimension. Such basic functions roughly correspond to membership functions in fuzzy logic. Each basic function is assigned a parameter w _i which is equal to the (expected) value of y given the valid rule i. In the network, this parameter corresponds to the initial weight. The output of the network is defined as follows:

In areas where only one rule is significant, the output is equal to w _i . In areas where more than one rule is significantly valid, the equation calculates a weighted average of the expenditure of the individual rules.

Fig. 1: Network topology

The topology of the neural network is shown in Figure 1. The operations of the nodes are given by equations (1) and (2). Each rule corresponds to a node in the middle layer of the network.

The network is changed by training data. The parameters (centers c _ÿ , widths σ _ÿ , weights w _i ) are modified by error back propagation. After learning, the adjusted rules can be extracted from the learned network using the same mechanism.

The structure of the rules and their incorporation into the neural network is as follows described. O.E.d.A a 2-dimensional entrance room is assumed. The 3 Different types of rules are designated with rule 1, rule 2 and rule 3. Rule ₁

if x 1 is A 1 1 (range 1 1 = R 1 1) and
if x₂ is A₂¹ (range₁₂ = R₂¹) then
y is B¹

corresponds to a Gaussian function centered around c₁₁ = A₁¹, c₁₂ = A₂¹ with σ₁₁ = R₁¹ and σ₁₂ = R₂¹ and assigned weight w₁ = B¹. Rule₂

if x₂ is A₁² (range₂₁ = R₁²)
then y is B²

represents a one-dimensional Gaussian function, centered around c₂₁ = A₁² with σ₂₁ = R₁² and weight w₂ = B². In this case it should be noted that the basic function is independent of x₂ is. Rules of the form like Regel₃

if x 1 is A 1 3 (range 3 1 = R 1 3) or
if x₂ is A₂³ (range₃₂ = R₂³) then
y is B³

are first transformed into the following 2 rules: Rule _3a

if x₁ is (A₁³ (range₃₁ = R₁³) then y is B³

and rule _3b

if x₂ is A₂³ (range₃₂ = R₂³) then y is B³

and then treated like Rule₂. With this procedure you can choose a network architecture construct a lot of rules. On the other hand, there is also a reverse approach possible, where you get a lot of simple rules from a network architecture again can pull out. With appropriate thinning processes for neural networks can Nodes are deleted and thus also their corresponding rules.  

If one follows the procedures described above, re gel-based knowledge used to advance the network structure and initialize, but with training the initial knowledge z. B. forget exponentially. The method has the advantage that the network starts with good starting conditions and good predictions gives up before training.

An alternative to the above procedure is to use the Pa freeze parameters of the network after following the Rules has been structured. In this case, too Additional basic functions for network adaptation are provided become. A procedure similar to that in J. Platt, A Resource-Allocating Network for Function Interpo lation. In: D.S. Touretzky (ed), Advances in Neural Infor mation Processing Systems 2, Kaufmann, 1990 follow, where new basic functions are introduced, whenever the network prediction differs significantly from the measured deviates from its initial values. In fact, anyone can new basic function interpreted as an additional rule become.

In a further alternative, a "penalty term" with fol gender form are introduced:

where q _{j is} a general network parameter (e.g. the weights of the network). This function is equivalent to that used to change weight when q _j initial = 0. α is a suitable penalty weight for the network parameters. B. is estimated by an expert.

Finally, another alternative is one To take a copy of the initialized network and accordingly to choose a "penalty term" with the following form

Ep can be approximated to

where Vunit is the volume of the unit grid and α is the has the meaning given above. Ep punishes the network if it wasn't with the initial network at the grid points matches. Instead of the grid points, reference points can be used be chosen at random or according to the centers of Basic function or the input training data. Inter Essentially, it can be said that instead of that Prior knowledge to use that directly through network parameters is defined, you can choose a specification that depends on of the mapping of what can be defined Network has to learn. This has the advantage that none Specification of relatively imprecise network parameters must be specified, but the specifications directly Reflect certainty that of the illustration of the in itialized network is assigned, and the often ge can be estimated.

For example according to M. Röscheisen, R. Hofmann, Volker Tresp, Neural Control for Rolling Mills: Incorporating Domain Theories to Overcome Data Deficiency. To be published in Advances in Neural Information Proces sing Systems 4, 1992 this certainty can be dependent be valued by problem specific knowledge.

Claims (6)

1. Neural network with a rule-based network structure, in which each output value y _{1 is} a normalized linear combination of I basic functions b _i (x ₁ ,..., x _N ) with I weighting coefficients w _i , the basic functions b _i (x ₁ ,..., x _N ) being non-negative functions localized by I mean values c _i of the N input values x ₁ ,. . . , x are _N. 2. Neural network according to claim 1 with basic functions of the form where x = (x ₁ ,..., x _N ), Σ _i ^{-1 is} the inverse covariance matrix and k _{i is} a normalization factor of the i-th basis function. 3. Method for structuring a neural network through rule-based knowledge, in which the basic functions of a neural network according to one of the preceding claims according to the type of membership functions for display fuzzy rules can be chosen.   4. Method for training a neural network according to one of the preceding claims,
in which the parameters of the network are frozen after the pre-structuring,
in which the deviation of the network from the initial state is simulated by a basic function that is used as a rule. 5. A method for training a neural network according to one of claims 1 to 3, wherein the deviation of the network from the initial state by the term: is simulated, where q _{j is} a network parameter (weights of the network), α is a penalty weight for the network parameters. 6. A method for training a neural network according to one of claims 1 to 3, wherein the deviation of the network from the initial state by the term using the approximation where α is a penalty weight for the network parameters, V _unit depends on the volume of the unit ^grid , x ^{grid is} the input value at a grid point or at randomly selectable points or at centers of the basic function or depending on initial training data.