JP2009277110A - Software test/development support device, and program for device - Google Patents

Abstract

[PROBLEMS] To improve the efficiency of a test process, it is possible to use information available before the test, such as design models and source code, to define test items and determine test priorities. However, the results could not be efficiently fed back to the determination of test items and test priorities.
A defect-related map analyzer 108 generates a defect-related map corresponding to an estimated value of the probability that two or more different development products have a defect at the same time. Uses the defect-related map to identify and display defect candidate-related information that is determined to have a high possibility of a defect based on information obtained by performing a test. Based on this display, the user changes the plan so that the test is preferentially performed from a location where there is a high possibility that a defect will be found.
[Selection] Figure 5

Description

  The present invention relates to a software test management technique for improving the efficiency of a software test process and improving the quality of software.

  The purpose of the software testing process in software development is to find software defects in the first place, in other words, to confirm that there are no software defects.

  However, it is generally impossible to inherit all behaviors of the developed software in tests, and it is often difficult in practice.

  Therefore, in the tests performed in software tests, the tests that can be performed are defined based on the design model, source code, or software requirements, and then the software tests are limited to the defined tests that can be performed. It is common.

  In order to streamline these test implementation processes, the management of deliverables such as design models, source code, test definitions or test results, in particular the management of information on the relationship and traceability of each deliverable, The process has been streamlined.

JP 2007-133800 A Kaner, Falk and Nguyen “Software Learning from the Basics” Nikkei Business Publications, 2001

  Conventionally, as in Patent Document 1, the efficiency of the test process has been improved by centrally and efficiently managing information related to testing of software to be developed.

  However, the definition of the items to be tested and the priority order of the items to be tested are left to the test designer's experience as described in Non-Patent Document 1, and are not the targets handled by the test management device. .

  For this reason, it is possible to use information available prior to the test, such as the design model or source code, to define the test items and to determine the priority of the test items. It was not possible to efficiently feed back to the determination of test items and test item priorities.

  The present invention was made to solve the above-described problems, and in addition to the information managed in the conventional test process, based on the information about the failure obtained as a result of performing the software test, Tests in the test plan by incorporating the detection of failure occurrence trends in the software to be developed into the process of the test management device and using this failure occurrence tendency to estimate the places where there is a high possibility of failure. The purpose is to positively feed back and utilize the information obtained from the test implementation in such a manner that the priority order of the items is changed.

  The subject of the present invention is an apparatus for managing information related to the planning and execution of tests in software development, comprising a source code storage device for recording source code as a first development product, and a test definition for the software to be developed. (2) a test definition storage device for storing as a development product, a test result storage device for storing a test result of the software to be developed as a third development product, and failure information representing a failure location of the software to be developed. A defect information storage device that records as a development product, a development product related information storage device that records at least related information between the first to fourth development products as development product related information, and at least the first ~ Estimated probability of including the case where two or more parts of the fourth development deliverables become defective at the same time A defect-related map storage device for storing a defect-related map to be represented, a data management server for managing each of the first to fourth development artifacts, each data of the development artifact-related information and the defect-related map, and an input The development results obtained by tracing the development product related information from the defects within the range of the test results that have been collected from the corresponding storage device via the data management server, respectively, A defect-related map analyzer for calculating a defect-related map before conversion by performing event extraction and performing the simultaneous defect-related event extraction at least once, and the defect-related map before conversion and its related data The defect related map obtained by collectively converting to a certain format, the defect via the data management server And having a fault-related map generation device for storing in the communication map store.

  According to the present invention, a defect-related map can be created on the basis of defect information of software to be developed discovered by a test. And using this defect-related map, it is possible to raise the priority of the test regarding the place where the possibility that the defect is lurking or the position where the serious defect is lurking, so that the discovery of the defect is accelerated, As a result, development efficiency can be improved.

  Hereinafter, various embodiments of the subject of the present invention will be described in detail along with the effects and advantages thereof with reference to the accompanying drawings.

(Embodiment 1)
FIG. 1 is a block diagram showing a hardware configuration of the software test / development support apparatus according to the present embodiment. As shown in FIG. 1, this apparatus includes a source code storage device 101, a test definition recording device 102, a test result storage device 103, a defect information storage device 104, a development product related information storage device 105, and a defect related map storage device. 106 is provided. Each of these storage devices 101 to 106 is realized using a magnetic disk or a nonvolatile semiconductor memory. The storage devices 101 to 106 are managed by the data management server 107 of this device. The data management server 107 controls the apparatus by a CPU (not shown) built in the server 107 to control signals between the storage devices 101 to 106 via an internal or external network. And transmission / reception of data signals.

  In addition, this apparatus includes a defect-related map analysis device 108 and a defect-related generation device 109. The defect-related map analysis device 108 and the defect-related generation device 109 are connected to the respective storage devices via the data management server 107. Data is input / output between 101-106. Each of the defect-related map analysis device 108 and the defect-related map generation device 109 that forms the core of the present embodiment includes a CPU (not shown) that performs calculation processing and data input / output control, and a temporary storage memory (see FIG. The operation of the microcomputer is controlled by a dedicated program, and control signals and data signals are exchanged with the data management server 107 via an internal or external network. Send and receive. In addition, a user input / output terminal (not shown) is connected to the defect-related map analysis device 108 and the defect-related generation device 109 via an internal or external network.

  Information stored in each of the storage devices 101 to 104 is generated as a result of activities in the process of software development. In other words, 1) source code is created as a result of an activity of implementation by a program, and 2) a test definition is an activity of implementation of a software test specification definition or a program for executing a software test. 3) The test result is a record of the result of the test implementation. In particular, the test results include an illegal behavior record that records software behavior that differs from what is assumed in the test definition. 4) The defect information is information obtained as a result of an activity of analyzing the cause based on the illegal behavior record. Hereinafter, these pieces of information are collectively referred to as “development products”.

  In the present embodiment, each piece of development product information is stored in the corresponding storage devices 101 to 104 as electronic data that conforms to a fixed description format. The specific description format may be a table format of a predetermined format or a more flexible format such as an XML format in which a schema is defined. Since the description format is determined in this way, the computer is provided with a means for specifying an arbitrary part of the development product by the granularity or unit of the description element defined by the format.

  Further, 5) development product related information is information determined at the stage where the development product is created by the dependency of each part of the development product. For example, a software test is performed based on a test definition that defines its contents, and as a result, a test result is obtained. Therefore, all test results depend on a specific test definition. Further, a relationship is established between the source code and the test definition that is a test target from the source code side and a test target from the test definition side. The defect information designates a part of the source code as a defect part, and is associated with the test result as an illegal behavior caused by a defect. Since the development product related information is described in a predetermined format and the description element can be specified, the description information or the link information between the sets is described in a fixed description format like the development product. It is generally expressed and stored in the development product related information storage device 105.

  Next, 6) the defect-related map is, as will be described later, generated by calculation by the defect-related map analysis device 108 based on the development product and the development product-related information, and the defect-related map. The information is converted into a certain format by the map generation device 109. Here, FIG. 2 is a diagram illustrating an example of the defect-related map. An example of FIG. 2 is a two-dimensional defect-related map that represents the relationship between test data on the vertical axis and programs on the horizontal axis. In FIG. 2, when the vertical axis is indicated by y and the horizontal axis is indicated by x, in the location of (x, y) on the defect-related map, when y is related to the defect, x is related to the defect. The estimated value of the probability is In that case, the table of FIG. 2 represents the probability as a percentage. Note that the prediction of the occurrence probability of a defect based on FIG. 2 will be described again in the third embodiment.

  The defect-related map in FIG. 2 is shown using information directly related to defects such as tests and programs on the vertical axis and the horizontal axis, respectively. Any classification is acceptable. Alternatively, the vertical axis and the horizontal axis may be the same classification. Alternatively, when both the vertical axis and horizontal axis classifications are not directly associated with the defect, the defect-related map analysis device 108 depends on the content determined by the development product-related information stored in the development product-related information 105. By following the chain of associations, association with defects is performed. Further, the defect-related map does not have to be two-dimensional data as shown in FIG. 2, and may be data in a high-dimensional space as necessary.

  Let X be a set of development artifacts arbitrarily designated, and x be an arbitrary subset thereof. For a certain illegal behavior, x represents an expected value including an element that causes the illegal behavior as P (x). If the direct cause of fraudulent behavior is not included in X, the direct cause of fraudulent behavior (here, y) is determined by following the relevant chain according to the contents defined by the development product information. If x is related, define P (x) = g (P (y)). However, g is a monotonically increasing value ranging from 0 to 1 and the specific form is a function determined by the strength of the relationship between x and y. If there is no relationship between x and y, P (x) = 0. The simultaneous expectation value P (x, y) is defined as an expectation value that includes the cause of incorrect behavior of x and y at the same time. For the sake of explanation, it is limited that there is only one illegal behavior. However, this may be extended with an arbitrary set of defects, and the set and extension of defects found by performing an arbitrary range of tests. You may do it. That is, P is an amount determined on the assumption of a certain defect or a set of tests.

  The values on the defect-related map are such that the vertical axis is x and the horizontal axis is y, and the conditional probability P (x | y) = P (x, y) / P (y), that is, y has the cause of the defect. This is an expected value that includes the cause of the failure in x under the condition of In a multi-dimensional defect-related map, a conditional probability P (y1,..., Where defects are found simultaneously in x1,..., Xm under conditions where defects are found simultaneously in y1, y2,. , yn | x1,..., xm) ≡ P (y1,..., yn, x1,..., xm) / P (x1,. y1, ..., yn, x1, ..., xm).

  Next, the theory of the calculation method of the defect-related map estimation value performed by the defect-related map analysis apparatus 108 will be described. Suppose that a defect or development product related (cause) from the defect is identified in parts x and y of the development product based on the illegal behavior observed by conducting an arbitrarily defined range of tests. . This is referred to as “simultaneous failure related event” of x and y. By varying the scope of the test, the frequency of simultaneous failure related events can be obtained. Similarly, a frequency is obtained in which x and y are independently associated with a defect. With these frequencies and the number of test ranges, the estimated value of P (y) is (the frequency y is associated with the defect alone) / (the number of test ranges from which the frequency was obtained) and P (x, The estimated value of y) is obtained as (frequency of simultaneous failure events of x, y) / (number of test ranges from which the frequency is obtained), respectively. Then, since the estimated value of the defect-related map is P (x | y) = P (x, y) / P (y), it is obtained by the defect-related map analyzer 108 by calculation based on the above relational expression.

  Next, a method of calculating the estimated value of the defect-related map using this apparatus will be described based on the flowchart of FIG. Note that steps 302 to 307 in FIG. 3 are operations in the defect-related map analyzer 108 controlled by the program.

  First, when the defect-related map analysis device 108 starts the calculation process of the defect-related map estimated value in response to the activation of the program (step 300), the user uses the input / output terminal (not shown). As a parameter to be used in the calculation of the estimated value, the test result range (310), the granularity of the development product specific unit (311), and the variables of the frequency and probability estimates are generally developed. The product set (312) is input / designated to the defect-related map analyzer 108 (step 301).

  In response to the input of the parameters, the defect-related map analysis apparatus 108 first determines the defect identified as the cause based on the test improper behavior within the designated test result range as a defect in the defect information storage device 104. Identify from the information, and further identify other development products related to the defect obtained by accessing the storage device 105 and tracing the development product related information in the storage device 105, and specifying it as related Each developed product is read from the corresponding storage devices 101 to 103 (step 302).

  Next, the defect-related map analysis device 108 obtains the frequency of simultaneous defect-related events for these development products on the defect side (step 303), and subsequently calculates the probability of simultaneous defect occurrence (step 304). After the calculation process, the defect-related map analysis apparatus 108 outputs the calculation result of step 304 to the input / output terminal and displays it to the user (the user sees the display result and obtains the expected result). Whether or not it is necessary to change the designated parameter), or automatically in the device 108 using a predetermined discriminant function in the device 108 The calculation result of 304 is evaluated, and it is determined whether or not the expected result has been obtained with the parameters up to that point (step 305), and the apparatus repeats steps 301 to 304 described above according to the determination result. .

  If it is determined by the automatic evaluation using the user or the discriminant function that the result obtained in step 304 is sufficient, the defect-related map analyzer 108 determines the defect related to the specified development product set part. The occurrence probability P (x) and the simultaneous failure occurrence probability P (x, y) are calculated (step 306), and the conditional probability P (x | y) is further calculated to determine the failure at each location (x, y). An estimated value of the related map is calculated (step 307). Then, the defect-related map analysis device 108 outputs the calculated defect-related map to the input / output terminal and displays it for evaluation by the user as necessary, or is determined in advance in the device 108. The evaluation is automatically performed based on the index, and the defect-related map generation device 109 summarizes the calculated defect-related map and the related data based on a storage instruction issued from the user or the device 108. The defect-related map is stored in the defect-related map storage device 106 after being converted into a predetermined format (step 309).

  Steps 306 and 307 may be performed immediately after step 304 or immediately before step 305.

  According to the present embodiment described above, development with a high probability that there is a defect that has not yet been found based on the illegal behavior observed by newly performing a test or the defect information obtained by analyzing it. It is possible to generate and store a defect-related map for predicting a development product associated with a deliverable or a defect having a high possibility of existence.

(Embodiment 2)
In the present embodiment, as shown in the block diagram of FIG. 4, 5) a design model such as a software requirement specification and design information held in the design model storage device 400, and 6) a development management data storage device 401 Recorded development management information at the time of development execution (for example, whether the user has skills, or when the user created software to be developed (whether busy) 4) is added to the development product in the first embodiment as a development product, and the defect-related map analysis apparatus 108 in FIG. Based on the method shown in FIG. 3 described in the first embodiment, a defect-related map including these development products 1) to 6) is created. The points other than the increase in the number of development products are the same as those in the first embodiment.

  According to the present embodiment, it is possible to use a design or specification defect upstream of the source code as viewed from the test specification as coordinates of the defect related map, and the development management information is included in the defect related map. It will be able to handle.

(Embodiment 3)
FIG. 5 is a block diagram showing a hardware configuration of the software test / development support apparatus according to the present embodiment. As shown in FIG. 5, the present apparatus has a feature point in that in addition to each component included in the apparatus (FIG. 1) of the first embodiment, the apparatus further includes a defect location estimation apparatus 500. Similarly to the defect-related map analysis device 108 and the defect-related map generation device 109, this defect location estimation device 500 is also an input / output terminal (not shown) that can be used by the user via an internal or external network. For example, a microcomputer. The defect location estimation apparatus 500 reads out the defect association map stored in the defect association map storage device 106 under the control of the dedicated program, and executes estimation of the defect location based on the defect association map by calculation. Presents a function. FIG. 6 is a flowchart showing an operation procedure of the defect location estimation apparatus 500 realized by the dedicated program. Note that this defect location estimation apparatus 500 may be applied to the apparatus of the second embodiment shown in FIG.

  When the failure location estimation apparatus 500 performs the failure location estimation operation based on FIG. 6, first, the user uses the input / output terminal to estimate the source (the location where the failure occurred from the test result and its The development product specifying unit or the defect-related product y related to the defect that is the cause of the defect is designated (step 601). In response to this input, the defect location estimation apparatus 500 loads y defect related maps from the defect related map storage device 106 (step 602). As described above, the defect-related map is tabular numerical data representing a conditional probability P (x | y) where x is an arbitrary development product. Next, the defect location estimation apparatus 500 searches for or identifies x such that the conditional probability P (x | y) for y is larger than the reference value using a predetermined reference value (step 603), a list of development products corresponding to the specified x is displayed on the input / output terminal (step 604). The defect location estimation apparatus 500 also displays related information managed by the development product access interface related to x on the input / output terminal (step 605).

  Assuming that the defect-related map illustrated in FIG. 2 is obtained, the defect location estimation apparatus 500 may estimate the following failure location based on the failure-related map. I can do it. In other words, if an incorrect behavior is observed as a result of performing the test of the test 1 data A which is y designated by the user, the defect location estimation apparatus 500, the program 1 or program in the defect-related map is detected. The conditional probability P (x | y) value 60 or 55 of 3 is compared with the reference value, and it is estimated that there is a high possibility that the program 1 or the program 3 is defective. In addition, in the illegal behavior of the test 1 data B, the failure location estimation apparatus 500 estimates that there is a high possibility that the failure is due to the failure of the program 2 or the program 3. By using the defect-related map in this way, the defect location estimation apparatus 500 can perform prediction regarding the failure location. Based on this prediction result, it is expected that the user will improve the development efficiency in the entire software development by determining the failure analysis or the priority order of the test items.

  Since the apparatus according to the present embodiment operates as described above, the user can use the generated defect-related map to generate development product information that is estimated to have a high probability of simultaneous defects from existing defect locations. Analyzes can identify potential defects. In other words, development to identify a predicted defect by predicting from the defect-related map a development product that has a high probability of having a defect that has not yet been found or that is associated with a defect that is highly likely to exist. Increase process priority. As a result, by conducting tests according to the priority order of the obtained test items, it becomes possible for the entire development to find defects at an earlier stage, and to improve development efficiency.

(Modification)
The defect-related map described in the first to third embodiments may be a defect-related map with importance to which an index representing the importance of corresponding data is added.

(Appendix)
While the embodiments of the present invention have been disclosed and described in detail above, the above description exemplifies aspects to which the present invention can be applied, and the present invention is not limited thereto. In other words, various modifications and variations to the described aspects can be considered without departing from the scope of the present invention.

It is a block diagram which shows the structure of the software test and development assistance apparatus which concerns on Embodiment 1 of this invention. It is a figure which shows the example of the malfunction relevant map which concerns on this invention. It is a flowchart which shows the production | generation method of the malfunction relevant map using the apparatus which concerns on Embodiment 1 of this invention. It is a block diagram which shows the structure of the software test and development assistance apparatus which concerns on Embodiment 2 of this invention. It is a block diagram which shows the structure of the software test and development assistance apparatus which concerns on Embodiment 3 of this invention. It is a flowchart which shows operation | movement of the malfunction location estimation apparatus which concerns on Embodiment 3 of this invention.

Explanation of symbols

  101 source code storage device, 102 test definition storage device, 103 test result storage device, 104 defect information storage device, 105 development product related information storage device, 106 defect related map storage device, 107 data management server, 108 defect related map analysis Device, 109 defect-related map generation device, 500 defect location estimation device.

Claims (5)

A device for managing information related to the planning and execution of tests in software development,
A source code storage device for recording the source code as a first development product;
A test definition storage device that stores the test definition of the software to be developed as a second development product;
A test result storage device for storing a test result of the software to be developed as a third development product;
A defect information storage device for recording defect information representing a defect portion of the software to be developed as a fourth development product;
A development product related information storage device for recording at least related information between the first to fourth development products as development product related information;
Expectation that at least two subsets of the development artifacts among the first to fourth development artifacts include a defect on the other condition under the condition that one of the development artifacts is included. A defect-related map storage device for storing a defect-related map representing a value;
A data management server for managing each of the first to fourth development deliverables, the development deliverable related information and the defect related map data;
Failures in the range of the entered test results and development artifacts obtained by tracing the development artifact related information from the defects are collected from the corresponding storage device via the data management server, and then simultaneously defected. A defect-related map analyzer for calculating a defect-related map before conversion by executing related event extraction and performing the simultaneous defect-related event extraction at least once;
The defect-related map for storing the defect-related map obtained by collectively converting the defect-related map and its related data before conversion into a fixed format in the defect-related map storage device via the data management server. A generating device,
Software testing and development support equipment. A software test and development support apparatus according to claim 1,
At least one of a design model storage device that records design information to be developed as a fifth development product and a development management data storage device that records development management information at the time of development execution as a sixth development product is the data. It is further provided as a storage device that manages the development product in the management server,
The development product related information recorded by the development product related information storage device includes related information on at least one of the fifth development product and the sixth development product,
The defect-related map includes two or more of at least one of the development products of the first to fourth development products, the fifth development product, and the sixth development product. It represents the estimated value of establishment including the case where the above parts become defective at the same time,
Software testing and development support equipment. The software test and development support apparatus according to claim 1 or 2,
Upon receiving the input of the development product y related to the defect that is the estimation source designated by the user, the defect related map for the defect development product y is received from the defect related map storage device via the data management server. By specifying a development product x having a conditional probability that is greater than a predetermined reference value from among the plurality of conditional probabilities P (x | y) represented by the read and read defect-related map, It further comprises a defect location estimation device that searches for a defect development product y and another development product x estimated to be highly related to the defect development product y as a defect candidate.
Software testing and development support equipment. Processing for executing the function of the defect-related map analyzer according to claim 1;
A process of executing the function of the defect-related map generation device according to claim 1;
It is characterized by having a computer execute.
Software testing and development support device program. The program according to claim 4,
The process of executing the function of the defect location estimation device according to claim 3,
In addition, the computer
Software testing and development support device program.

Images