CN107797931B - Software quality evaluation method and system based on secondary evaluation - Google Patents

Abstract

The invention provides a software quality evaluation method and a system based on secondary evaluation, which are characterized in that a software quality evaluation index space is selected, and a software quality evaluation result identification framework is constructed; collecting sample data and software data to be evaluated; determining the number and the topological structure of BP neural networks; training each BP neural network in parallel, and calculating the credibility; respectively inputting quality evaluation index data of software to be evaluated into each trained BP neural network, and obtaining a preliminary evaluation result according to the output result of the BP neural network; correcting the primary evaluation result by combining the reliability of the BP neural network to generate basic probability assignment allocation of each proposition in the recognition frame, and fusing each evidence by utilizing DS evidence theory to obtain a fusion result; and carrying out decision on the fusion result based on the decision criterion to generate a final evaluation result. The invention can effectively realize the evaluation of software quality.

Description

Software quality evaluation method and system based on secondary evaluation

Technical Field

The invention relates to the technical field of software quality evaluation, in particular to a software quality evaluation method and system based on secondary evaluation.

Background

Software quality is the sum of all features and all characteristics of a software product that are relevant to meeting a specified and implicit required capability. The software quality evaluation is to measure quality characteristics, sub-characteristics and the like of the evaluated software by using a proper technology and evaluate the measured result, thereby giving an evaluation result of whether the software product can meet specific requirements. The quality evaluation of the software can quantitatively give the quality level of the software, and the software can be correspondingly managed and improved according to the evaluation result data of the quality of the software.

At present, the software quality evaluation method mainly comprises the following steps:

1. analytical hierarchy process: and decomposing various factors influencing the quality of the software according to a certain standard to form different layers, and comparing the factors of the same layer by each expert in pairs to obtain a comparison matrix. However, when the comparison factors of the same level are more, the expert easily gets confused or contradictory judgment, and the comparison matrix is huge and has higher complexity.

2. Fuzzy comprehensive evaluation method: the evaluation standard of the index is subjected to fuzzy processing by adopting a method for constructing a membership function in fuzzy mathematics, and the maximum membership principle is utilized to ensure that the evaluation made according to the evaluation standard is more in line with the thinking mode of people. However, the defect of fuzzy mathematics can cause the principle to be frequently ineffective, so that phenomena such as evaluation result distortion, homogenization, jump and the like occur, and the selection of a proper membership function according to actual conditions is difficult.

3. Based on a support vector machine: the support vector machine has the unique advantage of processing the problem of small samples, avoids the process from induction to deduction of the traditional statistical learning method, and greatly simplifies the problems of traditional classification, regression and the like. However, the support vector machine is difficult to implement for large-scale training samples, and has a slightly poorer capability of solving the multi-classification problem.

4. Based on the neural network: the quality evaluation method for the complex software system is the quality evaluation method with highest current accuracy. Training the neural network by using the sample data, training the neural network as a software quality evaluation field expert, and inputting quality evaluation index data of the software to be evaluated into the neural network to obtain an evaluation result. However, the neural network is complex in calculation, the influence of parameter configuration on the output result is large, and particularly when the dimension is large, the convergence speed is low, and even the situations of non-convergence, erroneous judgment or missed judgment and the like can occur.

In summary, how to effectively implement the evaluation of the software quality is a problem to be solved.

Disclosure of Invention

In view of the above, the invention provides a software quality evaluation method and a system based on secondary evaluation, which can effectively solve the problems of low accuracy, larger influence by subjective factors of experts, weaker expandability and the like of the software quality evaluation method in the prior art.

The invention provides a software quality evaluation method based on secondary evaluation, which comprises the following steps:

selecting a software quality evaluation index space and constructing a software quality evaluation result identification framework;

collecting sample data according to a software quality evaluation index space and a software quality evaluation result identification framework, and collecting software quality evaluation index data of software to be evaluated;

determining the number of BP neural networks, and determining the topological structure of each BP neural network according to the software quality evaluation index space and the software quality evaluation result recognition frame;

for each BP neural network, dividing sample data into a training sample set and a test sample set, training the BP neural network by using the training sample set, and calculating the credibility of each BP neural network by using the test sample set;

respectively inputting quality evaluation index data of software to be evaluated into each trained BP neural network, and obtaining a preliminary evaluation result according to the output result of the BP neural network;

correcting the primary evaluation result by combining the reliability of the BP neural network to generate basic probability assignment allocation of each proposition in the recognition frame, and fusing each evidence by utilizing DS evidence theory to obtain a fusion result;

And carrying out decision on the fusion result based on the decision criterion to generate a final evaluation result.

Preferably, the determining the number of BP neural networks, according to the software quality evaluation index space and the software quality evaluation result recognition framework, determining the topology structure of each BP neural network includes:

determining the number L of BP neural networks;

determination of BP neural network C _i (i=1, 2, …, L);

and determining an activation function and a learning algorithm of the BP neural network.

Preferably, for each BP neural network, the dividing the sample data into a training sample set and a test sample set, training the BP neural network by using the training sample set, and calculating the credibility of each BP neural network by using the test sample set includes:

dividing the collected sample data set D into a training sample set and a test sample set by adopting a cross validation method;

respectively training BP neural network, and setting the currently trained BP neural network as C _i ；

Respectively calculating the credibility of each BP neural network, and setting the BP neural network which is currently processed as C _i 。

Preferably, the correcting the preliminary evaluation result by combining the reliability of the BP neural network to generate the basic probability assignment distribution of each proposition in the recognition frame, and fusing each evidence by using the DS evidence theory, wherein the obtaining the fusion result comprises:

Combining BP neural network C _i Reliability θ of (i=1, 2, …, L) _i For the preliminary evaluation results

Performing correction processing to generate evidence E _i Basic probability assignment E of (i=1, 2, …, L) _i ＝(m _i (A ₁ ),m _i (A ₂ ),…,m _i (A _N ),m _i (Θ)), wherein:

evidence E is obtained by DS evidence theory _i And (5) fusing to obtain a fusion result F.

A software quality evaluation system based on secondary evaluation, comprising:

the construction module is used for selecting a software quality evaluation index space and constructing a software quality evaluation result identification framework;

the acquisition module is used for acquiring sample data and acquiring software quality evaluation index data of the software to be evaluated according to the software quality evaluation index space and the software quality evaluation result identification frame;

the determining module is used for determining the number of BP neural networks and determining the topological structure of each BP neural network according to the software quality evaluation index space and the software quality evaluation result identification framework;

the calculation module is used for dividing sample data into a training sample set and a test sample set for each BP neural network, training the BP neural network by using the training sample set and calculating the credibility of each BP neural network by using the test sample set;

the output module is used for respectively inputting the quality evaluation index data of the software to be evaluated into each trained BP neural network, and obtaining a preliminary evaluation result according to the output result of the BP neural network;

The processing module is used for correcting the primary evaluation result by combining the reliability of the BP neural network, generating basic probability assignment allocation of each proposition in the recognition frame, and fusing each evidence by utilizing the DS evidence theory to obtain a fusion result;

and the generating module is used for making a decision on the fusion result based on the decision criterion to generate a final evaluation result.

Preferably, the determining module is specifically configured to:

determining the number L of BP neural networks;

determination of BP neural network C _i (i=1, 2, …, L);

and determining an activation function and a learning algorithm of the BP neural network.

Preferably, the computing module is specifically configured to:

dividing the collected sample data set D into a training sample set and a test sample set by adopting a cross validation method;

respectively training BP neural network, and setting the currently trained BP neural network as C _i ；

Respectively calculating the credibility of each BP neural network, and setting the BP neural network which is currently processed as C _i 。

Preferably, the processing module is specifically configured to:

combining BP neural network C _i Reliability θ of (i=1, 2, …, L) _i For the preliminary evaluation result R _i ⁺ Performing correction processing to generate evidence E _i Basic probability assignment E of (i=1, 2, …, L) _i ＝(m _i (A ₁ ),m _i (A ₂ ),…,m _i (A _N ),m _i (Θ)), wherein:

Evidence E is obtained by DS evidence theory _i And (5) fusing to obtain a fusion result F.

According to the technical scheme, the invention provides a software quality evaluation method based on secondary evaluation, when evaluating the software quality, firstly selecting a software quality evaluation index space, and constructing a software quality evaluation result identification frame; collecting sample data according to a software quality evaluation index space and a software quality evaluation result identification framework, and collecting software quality evaluation index data of software to be evaluated; determining the number of BP neural networks, and determining the topological structure of each BP neural network according to the software quality evaluation index space and the software quality evaluation result recognition frame; for each BP neural network, dividing sample data into a training sample set and a test sample set, training the BP neural network by using the training sample set, and calculating the credibility of each BP neural network by using the test sample set; respectively inputting quality evaluation index data of software to be evaluated into each trained BP neural network, and obtaining a preliminary evaluation result according to the output result of the BP neural network; correcting the primary evaluation result by combining the reliability of the BP neural network to generate basic probability assignment allocation of each proposition in the recognition frame, and fusing each evidence by utilizing DS evidence theory to obtain a fusion result; and carrying out decision on the fusion result based on the decision criterion to generate a final evaluation result. The software quality evaluation method disclosed by the invention has the advantages of higher accuracy, stronger expandability and shorter training time and evaluation time, and meets the requirement of quality evaluation on software.

Drawings

In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flow chart of an embodiment 1 of a software quality evaluation method based on secondary evaluation;

FIG. 2 is a schematic diagram of a method of dividing a sample data set into a training set and a testing set according to the present invention;

FIG. 3 is a flow chart of calculating the reliability of BP neural network according to the present disclosure;

FIG. 4 is a schematic structural diagram of an embodiment 1 of a software quality evaluation system based on secondary evaluation according to the present disclosure;

FIG. 5 is a block diagram of an on-board optoelectronic imaging system;

FIG. 6 is a training curve of the Levenberg-Marquard algorithm;

fig. 7 is a gradient descent algorithm training curve.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

As shown in fig. 1, a flowchart of an embodiment 1 of a software quality evaluation method based on secondary evaluation disclosed in the present invention is shown, where the method includes:

s101, selecting a software quality evaluation index space and constructing a software quality evaluation result identification framework;

let the selected software quality evaluation index space be u= { T ₁ ,T ₂ ,…,T _M }, T therein _i (i=1, 2, …, M) represents the i-th index in the software quality evaluation index space; the software quality evaluation result identification framework is Θ= { A ₁ ,A ₂ ,…,A _N And software evaluation results were N levels (V ₁ 、V ₂ 、…、V _N ) Proposition A _i (i=1, 2, …, N) means that the quality rating of the software currently under evaluation is V _i 。

S102, acquiring sample data according to a software quality evaluation index space and a software quality evaluation result identification framework and acquiring software quality evaluation index data of software to be evaluated;

s103, determining the number of BP neural networks, and determining the topological structure of each BP neural network according to the software quality evaluation index space and the software quality evaluation result identification framework;

the BP neural networks are mutually independent, and the topological structures and the parameter configurations of the BP neural networks can be completely different.

(1) And determining the number of BP neural networks as L.

(2) Determination of BP neural network C _i (i=1, 2, …, L).

(1) Number of hidden layers: set to 1. The study proves that the BP neural network with a single hidden layer and an activation function of Sigmoid can approach any complex function with any precision. Increasing the number of network layers can improve the calculation accuracy and reduce errors, but can lead to greater network complexity and increase training time. To improve the calculation accuracy, a hidden layer can be selected to be used to increase the number of nodes of the hidden layer.

(2) Number of nodes of input layer: equal to the number M of evaluation indexes in the evaluation index space U.

(3) Number of nodes of output layer: the number of propositions in the software quality evaluation result identification framework theta is equal to the number N of evaluation grade numbers of the evaluation results.

(4) Number of nodes in hidden layer: and determining the number of hidden layer nodes according to the combination of the empirical formula (1) and the learning training result, wherein i is the number of nodes of an input layer, o is the number of nodes of an output layer, and a is an integer in a [1,10] interval.

(3) And determining an activation function and a learning algorithm of the BP neural network.

The Sigmoid function is selected as the activation function, and the Levenberg-Marquard algorithm is selected as the learning algorithm.

S104, for each BP neural network, dividing sample data into a training sample set and a test sample set, training the BP neural network by using the training sample set, and calculating the credibility of each BP neural network by using the test sample set;

For each BP neural network, dividing sample data into a training sample set and a test sample set; training the BP neural network by using a training sample set, and calculating the credibility theta of the BP neural network by using a test sample set _i (i＝1,2,…,L)。

(1) The acquired sample data set D is divided into training sample sets and test sample sets by cross validation (cross validation). And D is divided into S mutually exclusive subsets with similar sizes according to a formula (2), and each subset keeps the consistency of data distribution as much as possible. And taking the union set of the S-1 subsets as a training sample set each time, taking the rest subsets as test sample sets, thus obtaining S groups of training sample sets and test sample sets, carrying out S times of training and S times of testing on the BP neural network, and calculating the credibility of the BP neural network according to S times of testing results. Without loss of generality, in the present invention, s=10, and the sample data set partitioning process is detailed in fig. 2.

(2) Respectively training BP neural network, and setting the currently trained BP neural network as C _i 。

(1) A training sample set is selected.

(2) Initializing parameters. Initializing and assigning weights and thresholds related in the BP neural network, and carrying out normalization processing on each sample data in the training set to enable each sample data to be located in a [ -1,1] interval.

(3) And calculating a new value obtained by inputting the weighted data in the input layer into the activation function in the hidden layer, and then performing weighted input processing on the new value to the output layer activation function to obtain a calculation result of the output layer.

(4) If the calculated result is in error with the expected result, the weight, threshold value are adjusted and the output result of each layer is recalculated until the error is within the error range.

(3) Respectively calculating the credibility of each BP neural network, and setting the BP neural network which is currently processed as C _i 。

And evaluating the test samples in the test sample set by using the trained BP neural network, so as to calculate the credibility of the BP neural network, and the credibility is shown in figure 3. The confidence level is used for correcting the primary evaluation result before fusing the primary evaluation result by using DS evidence theory.

Set BP neural network C _i The output result of (i=1, 2, …, L) is R _i ＝(C _i1 ,C _i2 ,…,C _iN ) Wherein C _ij Represent C _i Output result of j (j=1, 2, …, N) th node. Set Num _k The total number of samples, alpha, of the test sample set with the sequence number k _k Is C _i Evaluating correct test sample count，β _k Is C _i Evaluating the number of erroneous test samples, gamma _k Is C _i The number of test samples that gave the evaluation result was rejected, and it was apparent that this was true according to definition formula (3).

Num _k ＝α _k +β _k +γ _k (3)

In calculation C _i Before the credibility of (C), R needs to be set _i Normalization processing is carried out according to formulas (4) and (5), and the processing result is that

Wherein->

Is C _ij Normalized results, < >>

To calculate BP neural network C _i And (3) establishing an evaluation criterion for evaluating the software quality of the test sample by using only the BP neural network, wherein the evaluation criterion is shown in a formula (6). Wherein ε _i For a preset threshold value epsilon _i E (0, 1). Suppose C is to _i The normalized output result is that

If->

The following evaluation criteria are satisfied, the software quality evaluation result G' =v of the current test sample _p Otherwise, refusing to give the evaluation result, gamma _i The value is added to 1. If BP neural network C _i The given evaluation result is consistent with the real result of the test sample, alpha _i Value plus 1, otherwise beta _i The value is added to 1.

According to BP neural network C _i The BP neural network C is calculated according to formulas (7) and (8) for the processing condition of the whole test sample _i Reliability theta of output software quality preliminary evaluation result _i ，θ _i ∈[0,1]。

S105, respectively inputting quality evaluation index data of software to be evaluated into each trained BP neural network, and obtaining a preliminary evaluation result according to an output result of the BP neural network;

respectively inputting quality evaluation index data of software to be evaluated into BP neural network C _i (i=1, 2, …, L) to obtain an output result R _i R is taken as _i Normalization processing is carried out according to formulas (4) and (5) to obtain a preliminary evaluation result

Wherein->

Is C _ij Results after normalization treatment (C _i For A _j Is a preliminary evaluation result of (c).

S106, correcting the primary evaluation result by combining the reliability of the BP neural network to generate basic probability assignment allocation of each proposition in the recognition frame, and fusing each evidence by utilizing a DS evidence theory to obtain a fusion result;

(1) Combining BP neural network C _i Reliability θ of (i=1, 2, …, L) _i For the preliminary evaluation result R _i ⁺ Performing correction processing according to formula (9) to generate evidence E _i Basic probability assignment E of (i=1, 2, …, L) _i ＝(m _i (A ₁ ),m _i (A ₂ ),…,m _i (A _N ),m _i (Θ))。

(2) Evidence E is obtained by DS evidence theory _i And (5) fusing to obtain a fusion result F.

Let the possible value of random variable X be theta, call theta the identification framework of random variable X, if all elements in theta are mutually incompatible. In the invention, Θ= { A ₁ ,A ₂ ,…,A _N And software evaluation results were N levels (V ₁ 、V ₂ 、…、V _N ) Proposition A _i (i=1, 2, …, N) means that the quality rating of the software currently under evaluation is V _i . Let the recognition frame be Θ,2 ^Θ Is a power set of Θ if for the function m:2 ^Θ →[0,1]Satisfying equation (10), then m (A) is the base probability assignment for A.

Setting a function m 2 ^Θ →[0,1]To identify basic probability assignments on the framework Θ, the function BEL is called a trust function on Θ if the function BEL:2 ^Θ →[0,1]Equation (11) is satisfied and BEL (A) > 0, then A is the focal element of the trust function BEL.

The identification frame Θ is provided with trust functions BEL1 and BEL2, and the corresponding basic probability is assigned as m ₁ And m ₂ The corresponding focal element isA ₁ ,…,A _k And B ₁ ,…,B _r Then F in equation (12) is the collision factor and equation (13) is the Dempster combining rule. The Dempster combining rule satisfies the rule of combination, and when multiple evidences are fused, a pairwise fusion method can be adopted.

Evidence E for each evidence using Dempster combining rules _i (i=1, 2, …, L) to obtain a fusion result f= (m) _f (A ₁ ),m _f (A ₂ ),…,m _f (A _N ),m _f (Θ))。

And S107, making a decision on the fusion result based on a decision criterion to generate a final evaluation result.

The method for making decisions based on DS evidence theory fusion mainly comprises decisions based on trust functions, decisions based on basic probability assignment and decisions based on minimum risks. The invention selects a decision method based on basic probability assignment, and the decision criterion is shown in a formula (14). Let A be _p 、A _q If A _p All of the following criteria are satisfied, then A _p And (5) judging the result as a decision result, otherwise, refusing to give an evaluation result. Wherein Γ is ₁ 、Γ ₂ Is a preset threshold Γ ₁ ∈(0,1)，Γ ₁ E (0, 1). According to decision result A _p The final evaluation result G=V of the software to be evaluated can be known _p 。

As shown in fig. 4, a schematic structural diagram of an embodiment 1 of a software quality evaluation system based on secondary evaluation according to the present disclosure is shown, where the system includes:

the construction module 401 is used for selecting a software quality evaluation index space and constructing a software quality evaluation result identification framework;

let the selected software quality evaluation index space be u= { T ₁ ,T ₂ ,…,T _M }, T therein _i (i=1, 2, …, M) represents the i-th index in the software quality evaluation index space; the software quality evaluation result identification framework is Θ= { A ₁ ,A ₂ ,…,A _N And software evaluation results were N levels (V ₁ 、V ₂ 、…、V _N ) Proposition A _i (i=1, 2, …, N) means that the quality rating of the software currently under evaluation is V _i 。

The acquisition module 402 is used for acquiring sample data and acquiring software quality evaluation index data of the software to be evaluated according to the software quality evaluation index space and the software quality evaluation result identification framework;

a determining module 403, configured to determine the number of BP neural networks, and determine a topology structure of each BP neural network according to the software quality evaluation index space and the software quality evaluation result recognition framework;

The BP neural networks are mutually independent, and the topological structures and the parameter configurations of the BP neural networks can be completely different.

(1) And determining the number of BP neural networks as L.

(2) Determination of BP neural network C _i (i=1, 2, …, L).

(1) Number of hidden layers: set to 1. The study proves that the BP neural network with a single hidden layer and an activation function of Sigmoid can approach any complex function with any precision. Increasing the number of network layers can improve the calculation accuracy and reduce errors, but can lead to greater network complexity and increase training time. To improve the calculation accuracy, a hidden layer can be selected to be used to increase the number of nodes of the hidden layer.

(2) Number of nodes of input layer: equal to the number M of evaluation indexes in the evaluation index space U.

(3) Number of nodes of output layer: the number of propositions in the software quality evaluation result identification framework theta is equal to the number N of evaluation grade numbers of the evaluation results.

(4) Number of nodes in hidden layer: and determining the number of hidden layer nodes according to the combination of the empirical formula (1) and the learning training result, wherein i is the number of nodes of an input layer, o is the number of nodes of an output layer, and a is an integer in a [1,10] interval.

(3) And determining an activation function and a learning algorithm of the BP neural network.

The Sigmoid function is selected as the activation function, and the Levenberg-Marquard algorithm is selected as the learning algorithm.

The calculation module 404 is configured to divide the sample data into a training sample set and a test sample set for each BP neural network, train the BP neural network by using the training sample set, and calculate the credibility of each BP neural network by using the test sample set;

for each BP neural network, dividing sample data into a training sample set and a test sample set; training the BP neural network by using a training sample set, and calculating the credibility theta of the BP neural network by using a test sample set _i (i＝1,2,…,L)。

(1) The acquired sample data set D is divided into training sample sets and test sample sets by cross validation (cross validation). And D is divided into S mutually exclusive subsets with similar sizes according to a formula (2), and each subset keeps the consistency of data distribution as much as possible. And taking the union set of the S-1 subsets as a training sample set each time, taking the rest subsets as test sample sets, thus obtaining S groups of training sample sets and test sample sets, carrying out S times of training and S times of testing on the BP neural network, and calculating the credibility of the BP neural network according to S times of testing results. Without loss of generality, in the present invention, s=10, and the sample data set partitioning process is detailed in fig. 2.

(2) The BP neural network is trained respectively,let the currently trained BP neural network be C _i 。

(1) A training sample set is selected.

(2) Initializing parameters. Initializing and assigning weights and thresholds related in the BP neural network, and carrying out normalization processing on each sample data in the training set to enable each sample data to be located in a [ -1,1] interval.

(3) And calculating a new value obtained by inputting the weighted data in the input layer into the activation function in the hidden layer, and then performing weighted input processing on the new value to the output layer activation function to obtain a calculation result of the output layer.

(4) If the calculated result is in error with the expected result, the weight, threshold value are adjusted and the output result of each layer is recalculated until the error is within the error range.

(3) Respectively calculating the credibility of each BP neural network, and setting the BP neural network which is currently processed as C _i 。

And evaluating the test samples in the test sample set by using the trained BP neural network, so as to calculate the credibility of the BP neural network, and the credibility is shown in figure 3. The confidence level is used for correcting the primary evaluation result before fusing the primary evaluation result by using DS evidence theory.

Set BP neural network C _i The output result of (i=1, 2, …, L) is R _i ＝(C _i1 ,C _i2 ,…,C _iN ) Wherein C _ij Represent C _i Output result of j (j=1, 2, …, N) th node. Set Num _k The total number of samples, alpha, of the test sample set with the sequence number k _k Is C _i Evaluating the correct test sample count, beta _k Is C _i Evaluating the number of erroneous test samples, gamma _k Is C _i The number of test samples that gave the evaluation result was rejected, and it was apparent that this was true according to definition formula (3).

Num _k ＝α _k +β _k +γ _k (3)

In calculation C _i Before the credibility of (C), R needs to be set _i Normalization processing is carried out according to formulas (4) and (5), and the processing result is that

Wherein->

Is C _ij Normalized results, < >>

To calculate BP neural network C _i And (3) establishing an evaluation criterion for evaluating the software quality of the test sample by using only the BP neural network, wherein the evaluation criterion is shown in a formula (6). Wherein ε _i For a preset threshold value epsilon _i E (0, 1). Suppose C is to _i The normalized output result is that

If->

The following evaluation criteria are satisfied, the software quality evaluation result G' =v of the current test sample _p Otherwise, refusing to give the evaluation result, gamma _i The value is added to 1. If BP neural network C _i The given evaluation result is consistent with the real result of the test sample, alpha _i Value plus 1, otherwise beta _i The value is added to 1.

According to BP neural network C _i The BP neural network C is calculated according to formulas (7) and (8) for the processing condition of the whole test sample _i Output software quality preliminary evaluation result Degree of confidence theta _i ，θ _i ∈[0,1]。

The output module 405 is configured to input quality evaluation index data of software to be evaluated into each trained BP neural network, and obtain a preliminary evaluation result according to an output result of the BP neural network;

respectively inputting quality evaluation index data of software to be evaluated into BP neural network C _i (i=1, 2, …, L) to obtain an output result R _i R is taken as _i Normalization processing is carried out according to formulas (4) and (5) to obtain a preliminary evaluation result

Wherein->

Is C _ij Results after normalization treatment (C _i For A _j Is a preliminary evaluation result of (c).

The processing module 406 is configured to modify the preliminary evaluation result in combination with the reliability of the BP neural network, generate a basic probability assignment of each proposition in the recognition frame, and fuse each evidence by using the DS evidence theory to obtain a fused result;

(1) Combining BP neural network C _i Reliability θ of (i=1, 2, …, L) _i For the preliminary evaluation result R _i ⁺ Performing correction processing according to formula (6) to generate evidence E _i Basic probability assignment E of (i=1, 2, …, L) _i ＝(m _i (A ₁ ),m _i (A ₂ ),…,m _i (A _N ),m _i (Θ))。

(2) Evidence E is obtained by DS evidence theory _i And (5) fusing to obtain a fusion result F.

Let the possible value of random variable X be theta, call theta the identification framework of random variable X, if all elements in theta are mutually incompatible. In the invention, Θ= { A ₁ ,A ₂ ,…,A _N And software evaluation results were N levels (V ₁ 、V ₂ 、…、V _N ) Proposition A _i (i=1, 2, …, N) means that the quality rating of the software currently under evaluation is V _i . Let the recognition frame be Θ,2 ^Θ Is a power set of Θ if for the function m:2 ^Θ →[0,1]Satisfying equation (10), then m (A) is the base probability assignment for A.

Setting a function m 2 ^Θ →[0,1]To identify basic probability assignments on the framework Θ, the function BEL is called a trust function on Θ if the function BEL:2 ^Θ →[0,1]Equation (11) is satisfied and BEL (A) > 0, then A is the focal element of the trust function BEL.

The identification frame Θ is provided with trust functions BEL1 and BEL2, and the corresponding basic probability is assigned as m ₁ And m ₂ The corresponding focal element is A ₁ ,…,A _k And B ₁ ,…,B _r Then F in equation (12) is the collision factor and equation (13) is the Dempster combining rule. The Dempster combining rule satisfies the rule of combination, and when multiple evidences are fused, a pairwise fusion method can be adopted.

Evidence E for each evidence using Dempster combining rules _i (i=1, 2, …, L) to obtain a fusion result f= (m) _f (A ₁ ),m _f (A ₂ ),…,m _f (A _N ),m _f (Θ))。

The generating module 407 is configured to make a decision on the fusion result based on the decision criterion to generate a final evaluation result.

The method for making decisions based on DS evidence theory fusion mainly comprises decisions based on trust functions, decisions based on basic probability assignment and decisions based on minimum risks. The invention selects a decision method based on basic probability assignment, and the decision criterion is shown in a formula (14). Let A be _p 、A _q If A _p All of the following criteria are satisfied, then A _p And (5) judging the result as a decision result, otherwise, refusing to give an evaluation result. Wherein Γ is ₁ 、Γ ₂ Is a preset threshold Γ ₁ ∈(0,1)，Γ ₁ E (0, 1). According to decision result A _p The final evaluation result G=V of the software to be evaluated can be known _p 。

For further explanation of the present invention, specific analysis and test results will be described below.

An onboard photoelectric imaging system is selected as a complex software system to be evaluated, and is used for verifying the evaluation accuracy of the invention, wherein the system is shown in a detailed view in FIG. 5, and the related index data and the evaluation result level are shown in a detailed view in Table 1. The system mainly comprises on-ball main control software, in-ball main control software, servo controller software, image tracker software, visible light camera software, laser range finder software and infrared thermal imager software, and has the main functions of autonomous tracking imaging and multiple servo working modes such as manual searching, automatic searching, digital guiding, scanning and the like.

The method of the invention is realized by MATLAB R2014a and simulation experiments are carried out. Hardware configuration of the execution environment of MATLAB R2014 a: CPU is Intel Core i 5.2 GHz, memory is 4GB, hard disk is 1TB; the operating system is Win7SP1.

The following describes the software quality evaluation process of the on-board photoelectric imaging system by using the invention, and the values of various indexes are shown in table 1:

(1) And determining the input and output of the evaluation method.

The selected software quality evaluation index space is U= { T ₁ ,T ₂ ,…,T ₃₁ The process comprises the steps of } = { functional integrity, functional correctness, functional applicability, time characteristic, resource utilization, capacity, coexistence, interoperability, identifiable appropriateness, easy learning, easy operation, user error protection, user interface aesthetics, easy accessibility, maturity, availability, fault tolerance, easy recovery, confidentiality, integrity, anti-repudiation, accountability, authenticity, modularity, reusability, easy analysis, easy modification, easy testability, adaptability, easy installation, easy substitution }, and is detailed in table 1.

The built software quality evaluation result identification framework is Θ= { A ₁ ,A ₂ ,A ₃ ,A ₄ ,A ₅ The software quality evaluation results were classified into 5 classes, each using V ₁ 、V ₂ 、V ₃ 、V ₄ 、V ₅ To indicate that the software quality is excellent, good, acceptable, poor. Proposition A _i (i=1, 2,3,4, 5) means that the quality level of the currently-to-be-evaluated software is V _i 。

(2) 240 groups of evaluation data are collected as sample data according to the software quality evaluation index space, and the collected sample data are divided into a training sample set and a test sample set by adopting a cross validation method. For reasons of space limitations, 10 training samples and 4 test samples were selected to describe the present invention, as detailed in Table 1. Wherein the score of each evaluation index is 0,1, and the larger the score is, the better the quality of the software covered by the index is.

TABLE 1 training samples, test samples, software data to be evaluated

(3) After the software quality evaluation index space and the software quality evaluation result recognition frame are determined, the BP neural network C is determined _i (i=1, 2, …, 8), the network topology is detailed in table 2. The neural networks are mutually independent, and the topological structures and the parameter configurations of the neural networks can be completely different.

TABLE 2 topology of neural networks

With 1 st BP neural network C ₁ The process of training using the Levenberg-Marquard algorithm is described for purposes of example. BP neural network C ₁ The parameters of (a) are set as follows: the learning rate is 0.1, the maximum training frequency is 100000, the error precision is 0.000001, the activation function is a Sigmoid function, the learning algorithm is respectively set to be a Levenberg-Marquard algorithm and a gradient descent algorithm, other parameters are set to be the same, then sample data in a training set are input, the BP neural network is trained, and corresponding training curves are shown in fig. 6 and 7 in detail. The analysis experiment result shows that the training effect of the Levenberg-Marquard algorithm is obviously better than that of the gradient descent algorithm, the gradient descent algorithm reaches the error precision after 34376 iterations, and the Levenberg-Marquard algorithm reaches the error precision after 9 iterations.

(4) After the BP neural network training is completed, the trained 8 BP neural networks are used for evaluating the test samples respectively, and the credibility of the 8 BP neural networks is calculated and obtained, and is shown in Table 3 in detail.

(5) And (3) respectively inputting M=31 evaluation indexes in U into L=8 BP neural networks to perform preliminary evaluation on software quality. The complex software system was primarily evaluated using the above-described trained 8 BP neural networks, respectively, and the results are detailed in table 3. If only one BP neural network is used for evaluating software qualityThe value (see formula (6)) is directly based on

The determined software quality evaluation result is G', and analysis table 3 shows that C ₃ And C ₅ Refusing to give the evaluation result, C ₂ The determined software quality grade is V ₃ ，C ₁ 、C ₄ 、C ₆ 、C ₇ And C ₈ The determined software quality grade is V ₂ The evaluation conditions of the BP neural networks differ greatly, so that fusion using DS evidence theory is necessary to mitigate uncertainty.

(6) Output result R of BP neural network _i (i=1, 2, …, 8) respectively and sequentially performing normalization processing and utilizing the credibility theta _i Correction processing, generating evidence E _i The assignment of basic probability assignments to the propositions in the recognition framework is detailed in Table 4.

(7) The results D of the 8 evidences fused by DS evidence theory are shown in Table 5.

(8) Setting a threshold in the decision criterion to Γ ₁ ＝0.10，Γ ₂ =0.10, the decision result according to the decision criterion is a ₂ I.e. the final evaluation result of a certain onboard photoelectric imaging system to be evaluated is g=v ₂ (good) and the true evaluation results in the sample data are consistent.

TABLE 3 preliminary evaluation results, direct evaluation results

Table 4 basic probability assignment for identifying propositions in a framework

Table 5 DS evidence theory fusion results

In order to verify the evaluation accuracy, training time and evaluation processing time of the method of the present invention, the above 240 sets of evaluation data were processed by the typical methods of the present invention and table 6, respectively, and the experimental results are shown in tables 7 and 8. Wherein, the evaluation processing time refers to the time from inputting the evaluation index data of the software to be evaluated into the evaluation method to obtaining the software quality evaluation result.

Table 6 experimental involves an overview of the evaluation method

TABLE 7 evaluation results of software to be evaluated (data sequence number 15)

Table 8 performance index comparison

Evaluation method	Accuracy of evaluation results	Training time	Evaluation of processing time
				PNN	73.86％	183.80ms	73.50ms
BPNN	76.73％	48285.00ms	76.60ms
				LM-BPNN	85.16％	553.20ms	77.10ms
RBFNN	89.75％	201.50ms	70.80ms
				The invention is that	95.23％	576.00ms	77.50ms

As can be seen from analysis of tables 7 and 8, the evaluation accuracy of the method of the invention can reach 95.23%, and is improved by 6.11% compared with RBFNN with better evaluation accuracy. The reason for this is:

(1) Compared with other methods, the method provided by the invention has the advantages that on the basis of fully playing the characteristics of the BP neural network to perform preliminary evaluation on the software quality, the DS evidence theory is utilized again to fuse the preliminary evaluation results, so that the uncertainty of evaluation data is further weakened, and the interference and conflict between evaluation indexes are effectively solved.

(2) In view of the flexibility of the system structure, each BP neural network can be dynamically modified and configured according to the credibility condition of the BP neural network so as to further improve the evaluation accuracy.

The training time of the method is 576.0ms, which is reduced by 98.81% compared with the BPNN adopting a gradient descent learning algorithm, and is the same as the LM-BPNN training time. The evaluation processing time of the method is 77.5ms, and is only increased by 9.46% compared with RBFNN with the highest evaluation speed. The reasons for this are:

(1) The invention uses a plurality of BP neural networks, but the BP neural networks are mutually independent, the training and preliminary evaluation of each BP neural network can be processed in parallel, and the training time and the preliminary evaluation time of the BP neural network only depend on one or more BP neural networks with the longest training time and the longest preliminary evaluation time. In addition, in view of the flexibility of the system structure, the BP neural network can be dynamically modified and configured according to the training time and the preliminary evaluation time of the BP neural network so as to further reduce the training time and the evaluation time of the method.

(2) Compared with other methods, the method only increases the processing time of carrying out data fusion by the DS evidence theory, but the DS evidence theory has the advantages of higher data fusion speed and lower time cost.

In summary, the innovation points of the invention are as follows:

1. the BP neural network is utilized to solve the problem that the evaluation result is greatly influenced by subjective factors of experts. The relation between the BP neural network input and the BP neural network output is a highly nonlinear mapping relation, the relation is difficult to express generally, the objective weight is automatically obtained according to feedback information through a training and learning process, and the influence of expert subjective factors in the traditional evaluation method is greatly weakened. Meanwhile, the parameters of the BP neural network have a larger influence on the fusion result, and when the prior knowledge of the problem is lacking, a large number of experiments are required to be searched for to determine the proper parameter setting. In order to solve the problem, the invention adopts a plurality of BP neural networks to carry out independent preliminary evaluation on the same evaluation index space, and utilizes DS evidence theory to fuse the preliminary evaluation results so as to obviously improve the generalization capability of the BP neural networks.

2. Interference and conflict among evaluation indexes can be effectively solved. The invention solves the problem by fusing the preliminary evaluation results by using DS evidence theory. Meanwhile, the problem that the DS evidence theory gives out paradox when the evidence of high collision is fused is fully considered.

3. In the invention, BP neural networks are mutually independent, and the topological structures and parameter configurations of the BP neural networks can be completely different, so that the system structure has the following advantages:

(1) The training and preliminary evaluation of each BP neural network can be processed in parallel, the parallel processing capacity of a computer can be fully utilized, and the data processing speed is further increased;

(2) The evaluation method has flexible system structure and strong expandability. One or more BP neural networks can be conveniently configured, added, deleted and modified, and because other BP neural networks are not affected, the existing trained BP neural networks do not need to be retrained and the reliability of the output software quality preliminary evaluation result (hereinafter referred to as the reliability of the BP neural network) is calculated, and the newly added and modified BP neural networks can be directly integrated into the existing evaluation method after the reliability is calculated. Meanwhile, the BP neural network can be dynamically modified and configured according to the training speed, the preliminary evaluation speed and the credibility condition of the BP neural network so as to further improve the training speed, the evaluation processing speed and the evaluation accuracy.

In the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.

Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software modules may be disposed in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (4)

1. The software quality evaluation method based on the secondary evaluation is characterized by comprising the following steps of:

selecting a software quality evaluation index space and constructing a software quality evaluation result identification framework;

collecting sample data according to a software quality evaluation index space and a software quality evaluation result identification framework, and collecting software quality evaluation index data of software to be evaluated;

determining the number of BP neural networks, and determining the topological structure of each BP neural network according to the software quality evaluation index space and the software quality evaluation result recognition frame;

for each BP neural network, dividing sample data into a training sample set and a test sample set, training the BP neural network by using the training sample set, and calculating the credibility of each BP neural network by using the test sample set, wherein the method comprises the following steps: the sample dataset D is formulated as:

dividing the BP neural network into S mutually exclusive subsets with similar sizes, taking a union set of S-1 subsets as a training sample set each time, taking the rest subsets as a test sample set, thus obtaining S groups of training sample sets and test sample sets, carrying out S times of training and S times of testing on the BP neural network, and calculating the credibility of the BP neural network according to S times of testing results;

The quality evaluation index data of the software to be evaluated is respectively input into each trained BP neural network, and a preliminary evaluation result is obtained according to the output result of the BP neural network, and the method comprises the following steps:

obtaining a preliminary evaluation result by the following formula

C _ij Representing BP neural network C _i Output results of j (j=1, 2, …, N) th node;

correcting the primary evaluation result by combining the reliability of the BP neural network to generate basic probability assignment allocation of each proposition in the recognition frame, and fusing each evidence by utilizing DS evidence theory to obtain a fusion result;

based on the decision criterion, making a decision on the fusion result to generate a final evaluation result;

the method for calculating the reliability of the BP neural network according to the S test results comprises the following steps:

the following formula is employed as an evaluation criterion when evaluating software quality of a test sample using only a BP neural network, wherein,

BP neural network C respectively _i Normalized output result, ε _i For a preset threshold value epsilon _i E (0, 1); if->

If the following formula is not satisfied, refusing to give an evaluation result;

the reliability of the BP neural network is calculated according to the following formula,

Num _k ＝α _k +β _k +γ _k ；

wherein θ _i Is the credibility of BP neural network, theta _i ∈[0,1]；Num _k The total number of samples, alpha, of the test sample set with the sequence number k _k Evaluating the correct test sample number, beta, for BP neural network _k Evaluating the number of erroneous test samples for BP neural network, gamma _k Rejecting the number of test samples giving the evaluation result to the BP neural network;

the determining the number of the BP neural networks and determining the topological structure of each BP neural network according to the software quality evaluation index space and the software quality evaluation result recognition frame comprises the following steps:

determining the number L of BP neural networks;

determination of BP neural network C _i (i=1, 2, …, L);

wherein the number of hidden layers in the topology is set to 1,the number of nodes of the input layer is equal to the number M of the evaluation indexes in the evaluation index space U, the number of nodes of the output layer is equal to the number of propositions in the software quality evaluation result identification framework theta, namely the number N of evaluation grades of the evaluation result, and the number of nodes of the hidden layer is according to a formula

Is determined by combining the learning training result, wherein i is the node number of the input layer, o is the node number of the output layer, and a is [1,10 ]]Integers within the interval;

determining an activation function and a learning algorithm of the BP neural network;

the primary evaluation result is corrected by combining the reliability of the BP neural network, basic probability assignment allocation of each proposition in the recognition frame is generated, each evidence is fused by utilizing DS evidence theory, and the acquisition of the fusion result comprises the following steps:

Combining BP neural network C _i Reliability θ of (i=1, 2, …, L) _i For the preliminary evaluation results

Performing correction processing to generate evidence E _i Basic probability assignment E of (i=1, 2, …, L) _i ＝(m _i (A ₁ ),m _i (A ₂ ),…,m _i (A _N ),m _i (Θ)), wherein:

evidence E is obtained by DS evidence theory _i And (5) fusing to obtain a fusion result F.

2. The method of claim 1, wherein for each BP neural network, dividing the sample data into a training sample set and a test sample set, training the BP neural network using the training sample set, and calculating the reliability of each BP neural network using the test sample set comprises:

dividing the collected sample data set D into a training sample set and a test sample set by adopting a cross validation method;

respectively training BP neural network, and setting the currently trained BP neural network as C _i ；

Respectively calculating the credibility of each BP neural network, and setting the BP neural network which is currently processed as C _i 。

3. A software quality evaluation system based on secondary evaluation, comprising:

the construction module is used for selecting a software quality evaluation index space and constructing a software quality evaluation result identification framework;

the acquisition module is used for acquiring sample data and acquiring software quality evaluation index data of the software to be evaluated according to the software quality evaluation index space and the software quality evaluation result identification frame;

The determining module is used for determining the number of BP neural networks and determining the topological structure of each BP neural network according to the software quality evaluation index space and the software quality evaluation result identification framework;

the calculation module is configured to divide sample data into a training sample set and a test sample set for each BP neural network, train the BP neural network by using the training sample set, and calculate the credibility of each BP neural network by using the test sample set, and includes: the sample dataset D is formulated as:

dividing the BP neural network into S mutually exclusive subsets with similar sizes, taking a union set of S-1 subsets as a training sample set each time, taking the rest subsets as a test sample set, thus obtaining S groups of training sample sets and test sample sets, carrying out S times of training and S times of testing on the BP neural network, and calculating the credibility of the BP neural network according to S times of testing results;

the output module is used for respectively inputting the quality evaluation index data of the software to be evaluated into each trained BP neural network, and obtaining a preliminary evaluation result according to the output result of the BP neural network, and comprises the following steps:

obtaining a preliminary evaluation result by the following formula

C _ij Representing BP neural network C _i Output results of j (j=1, 2, …, N) th node;

The processing module is used for correcting the primary evaluation result by combining the reliability of the BP neural network, generating basic probability assignment allocation of each proposition in the recognition frame, and fusing each evidence by utilizing the DS evidence theory to obtain a fusion result;

the generating module is used for making a decision on the fusion result based on the decision criterion to generate a final evaluation result;

the method for calculating the reliability of the BP neural network according to the S test results comprises the following steps:

the following formula is employed as an evaluation criterion when evaluating software quality of a test sample using only a BP neural network, wherein,

BP neural network C respectively _i Normalized output result, ε _i For a preset threshold value epsilon _i E (0, 1); if->

If the following formula is not satisfied, refusing to give an evaluation result;

the reliability of the BP neural network is calculated according to the following formula,

Num _k ＝α _k +β _k +γ _k ；

wherein θ _i Is the credibility of BP neural network, theta _i ∈[0,1]；Num _k The total number of samples, alpha, of the test sample set with the sequence number k _k Evaluating the correct test sample number, beta, for BP neural network _k Evaluating the number of erroneous test samples for BP neural network, gamma _k Rejecting the number of test samples giving the evaluation result to the BP neural network;

the determining module is specifically configured to:

Determining the number L of BP neural networks;

determination of BP neural network C _i (i=1, 2, …, L);

wherein the number of hidden layers in the topology structure is set to be 1, the number of nodes of the input layer is equal to the number M of evaluation indexes in the evaluation index space U, the number of nodes of the output layer is equal to the number of propositions in the software quality evaluation result identification framework Θ, namely the number N of evaluation grades of the evaluation result, and the number of nodes of the hidden layer is according to a formula

Is determined by combining the learning training result, wherein i is the node number of the input layer, o is the node number of the output layer, and a is [1,10 ]]Integers within the interval;

determining an activation function and a learning algorithm of the BP neural network;

the processing module is specifically configured to:

combining BP neural network C _i Reliability θ of (i=1, 2, …, L) _i For the preliminary evaluation results

Performing correction processing to generate evidence E _i Basic probability assignment E of (i=1, 2, …, L) _i ＝(m _i (A ₁ ),m _i (A ₂ ),…,m _i (A _N ),m _i (Θ)), wherein:

evidence E is obtained by DS evidence theory _i And (5) fusing to obtain a fusion result F.

4. A system according to claim 3, wherein the calculation module is specifically configured to:

dividing the collected sample data set D into a training sample set and a test sample set by adopting a cross validation method;

Respectively training BP neural network, and setting the currently trained BP neural network as C _i ；

Respectively calculating the credibility of each BP neural network, and setting the BP neural network which is currently processed as C _i 。

Images