CN100451989C - Software testing system and testing method - Google Patents

Abstract

This invention relates to one software test method, which comprises the following steps: analyzing test leg and establishing test sample; accordingly getting the test data; processing test machine according to the match; according to the match result fixing test environment; executing test spot and collecting test record and dismantling the test environment. This invention software test system comprises the following parts: control machine to analyze test leg and to establish test sample to adjust test machine and to send the data into the test machine fix environment and to record result and to collect test data according to the orders; match machine to match needed machines; test machine is to fix environment and to send order for relative aim.

Description

Software testing system and testing method

technical field

The invention relates to software testing technology, in particular to a software testing system and a testing method capable of automatic testing of software.

Background technique

Testing is one of the important links in the software development process, and sufficient testing is an important basis for ensuring software reliability. In recent years, with the development of software, the scale of software has continued to expand, and the structure of software deployment has gradually become distributed.

Software testing has the following characteristics: each regression test of large-scale software needs to execute a huge number of test cases, thus, software testing requires more computing power; testing requires different types of different versions of hardware, operating systems and dependent libraries The test needs not only enough times, but also needs to be tested in a wide enough environment; the complexity of modern systems almost makes each environment unique, and the software implementation environment, especially the configuration, is often different from the development environment. The environment is very different. Only by testing in a variety of real environments can we capture configuration errors and defects that cannot be covered in the development environment. Software testing requires a more realistic testing environment; distributed software can only work when multiple components work together , testing such software needs to be installed and configured on multiple machines. In addition, there are many modern software testing methods and tools, and a software test often needs to combine these methods and tools. This requires software testing to be able to install, configure and The combination of test tools is more automated.

Existing software testing techniques are generally carried out on a single machine or in a test environment composed of several machines, which requires a lot of human intervention. Using the existing software testing technology for grid software testing, the testing time is long, the efficiency is low, the testing environment is too single, and it is difficult to fully test the software, so the existing software testing technology can no longer meet the needs of software testing.

Grid computing can effectively aggregate large-scale resources on the Internet to perform high-performance parallel computing and collaborative work. The high degree of parallelism of the grid can speed up test execution and increase productivity. The grid is a highly heterogeneous environment. The richness of resources can provide a variety of test environments for testing, reducing the cost of acquisition and maintenance of test environments, and these environments are closer to the real operating environment of the software. The synergy of the grid allows machines distributed all over the world to jointly complete a test task.

Contents of the invention

The purpose of the present invention is to provide a software testing method and system to improve the efficiency of software testing.

In order to achieve the above object, the present invention provides a software testing method, comprising the following steps:

Analyze the test script and set up a test instance; the test data description, the test machine configuration description and the test scene are recorded in the test script; the test data is obtained according to the test data description; the test machine is matched according to the test machine configuration description; if the match is successful, pass The data service provided by the data service equipment of the grid transmits test data, installs the test environment, and if the matching fails, judge whether the number of matching times reaches the maximum value, if so, end; otherwise, wait for a predetermined time to receive the matching result, and judge whether it is successful; execute the test Scenarios, collect test records, and uninstall the test environment.

Described execution test scene, collect test record, the step of unloading test environment specifically comprises: control machine obtains an unexecuted scene in the test scene list, sends to the test machine the instruction of executing the operation target, and according to the operation result that test machine returns to The testing machine sends an instruction to execute the target of the running result; receives the execution result and registers the test result; judges whether all test scenarios have been executed, and if not, executes the running target; when each scenario is executed, the control machine sends a completion command to the testing machine; receives Test records; the testing machine executes corresponding operations and returns execution results according to the received instructions; if the instructions are completion instructions, then send test records to the control computer and unload the test environment; if the instructions are termination instructions, then unload Test environment; if the instruction is an instruction to execute an operation target or an instruction to execute an operation result object, the execution result is sent to the control machine after execution; if an exception occurs during execution, the exception is reported to the control machine.

The test machine can also register attribute information with the grid service information device, publish function information and local software list, so that other devices can easily know the attribute information of the registration machine, the function service provided and the existing software.

In order to achieve the above object, the present invention also provides a software testing system, the system includes: a control machine, used to analyze the test script, establish a test instance, call the matching machine to select the test machine, transmit the test data to the test machine and Install the test environment on the test machine, send instructions to the test machine according to the test scene, record the test results, and collect the test records on each test machine; the matching machine is used to match the test machine required for testing; the test machine is used to install the test environment, and according to The command sent by the control machine executes the corresponding operation.

The control machine may specifically include: a parsing module for parsing test scripts and establishing a test instance module; a data transmission module for transmitting test data through grid data service equipment; a matching service client for communicating with the matching machine , call the matching machine to select the test machine; the test machine service client is used to communicate with the test machine, transmit the test data and instructions to the test machine and receive the execution results, test results and test records returned by the test machine; the storage library is used for Store test results and test records.

The matching machine may specifically include: an information collection module, used to obtain the attribute information of the testing machine from the information service equipment of the grid; an explanation module, used to judge whether the testing machine meets the test requirements; a matching service module, used to communicate with the control machine Communication, receiving the matching request sent by the control machine, and returning the matching result to the control machine.

The testing machine may specifically include: a data transmission module, for transmitting test data; a test service module, for communicating with the control machine, receiving instructions and returning execution results; a test execution module, for performing corresponding operations according to instructions; information registration Module, used to register attribute information to the information service device of the grid.

The testing machine may further include: a dependency library management module, which is used to maintain the local software of the testing machine and publish the software list to the information service equipment of the grid; thus, for the software often needed in the testing process, it is not necessary to Transmission and installation, thereby reducing network transmission and further improving test efficiency. The test machine can further include a test function management module, which is used to manage the test functions provided by the test machine, and publish the test function information to the information service equipment of the grid, so that other devices can learn the test service functions provided by the test machine , to facilitate the matching selection of the testing machine and the effective testing.

The testing machine may further include: a safety module, which is used to ensure the safety of the testing machine during the testing process, and avoid things that threaten the safety of the testing machine.

The technical scheme of the present invention uses the grid to test the software, and the high parallelism of the grid can accelerate the execution of the test and improve the test efficiency; the grid is a highly heterogeneous environment, and the richness of its resources can provide a variety of tests for the test. The test environment reduces the acquisition and maintenance cost of the test environment, and these environments are closer to the real operating environment of the software; the synergy of the grid allows machines distributed in various places to jointly complete a test task and improve resource utilization.

The technical solutions of the present invention will be described in further detail below with reference to the accompanying drawings and embodiments.

Description of drawings

Fig. 1 is a schematic structural diagram of a software testing system embodiment one of the present invention;

Fig. 2 is a schematic structural diagram of the second embodiment of the software testing system of the present invention;

Fig. 3 is a schematic structural diagram of the third embodiment of the software testing system of the present invention;

Fig. 4 is a schematic structural diagram of a fourth embodiment of the software testing system of the present invention;

Fig. 5 is a flow chart of the first embodiment of the software testing method of the present invention;

Fig. 6 is the execution flowchart of the control machine of the second embodiment of the software testing method of the present invention;

FIG. 7 is a flow chart of the execution of the testing machine in Embodiment 2 of the software testing method of the present invention.

Detailed ways

As shown in FIG. 1 , it is a structural schematic diagram of Embodiment 1 of the software testing system of the present invention. The control machine controls multiple test machines to complete the test collaboratively, parses the test script, establishes a test instance, calls the matching machine to select the test machine, transmits the test data to the test machine and installs the test environment on the test machine, and sends instructions to the test machine according to the test scenario , record test results, and collect test data. The test often needs to be carried out under a specified platform, operating system or configuration, and the function of the matching machine is to retrieve a suitable test machine for the test from the grid environment. After the test machine is selected, it receives the test data sent by the control machine, installs the test environment, and performs corresponding operations according to the instructions sent by the control machine. Both the control machine and the test machine are dynamically selected from the grid environment. In this embodiment, the control machine controls a plurality of test machines to cooperate to complete the test, which can effectively improve the test efficiency; since the test machines are dynamically selected, the software can be tested in various test environments.

The test script in this system includes a series of goals, and each goal includes a series of tasks, and these tasks are combined to complete the goal. In this system, test scripts are divided into two types: master script and slave script:

The main script is executed by the control machine, which contains the description of the test data, the description of the test machine configuration and a set of test scenarios. The description of the test data is used to specify the test data required by the test machine during the test process. The test data includes the source code of the software, configuration files, executable files of the software, and other data related to the test. The description of the test machine configuration is used to specify what kind of configuration machine is required as the test machine. Usually a test needs to specify configuration information such as platform to run, operating system, CPU size, memory size, storage capacity, storage rate and grid bandwidth, etc. During the execution of the script, the system will dynamically select the appropriate testing machine from the grid environment according to the information. A test scenario is a description of a test task that includes a specific set of steps with an expected result. A test scenario consists of a "run" target, a "success" target, a "failure" target, and an "error" target. The "run" goal contains the steps needed to complete a test scenario. The latter three goals are determined by the execution result of the "run" goal which will be executed. The "success" goal will execute if the result is pass, and the "failure" goal will execute if the result is fail. If an exception occurs during the execution of the "run" object, the "error" object will be executed. The "success" goal includes assertions that pass if all of them are true and fail if one is false.

The slave script is loaded and executed by the control machine to control the test machine. The target in the slave script will be executed by the relevant tasks in the main script by sending corresponding instructions to the test machine through the control machine, and the test machine will return the execution result to the control machine after execution.

The control machine interprets and executes the main test script, which is the brain of the test process. For each main test script, it calls the matching machine to select a set of test machines from the grid environment. It coordinates the coordination and synchronization between various testing machines. The test code is not executed on this machine. During the execution of the test process, it is responsible for obtaining test data and transmitting it to each test machine, collecting test status and results, and transferring information between test machines.

The matching machine is responsible for matching the testing machines needed for testing from the grid environment. In this system, each test machine is described by a group attribute. These attributes include: CPU speed, memory size, hard disk size, hard disk speed, operating system type, etc. and test-related configurations. These information are all registered in the information service device of the grid. When the matching machine receives a matching request, it will obtain the attributes of the testing machine from the information service device and cache them. An expression can be used to describe whether the test machine meets the requirements. The attribute of the test machine is a variable that matches the expression in the request. If the operation result of the expression is true, it means that the test machine meets the requirements.

The testing machine is initialized and controlled by the control machine, which is mainly responsible for accepting the commands sent by the control machine and performing corresponding operations. These commands include: data transfer, load from script, execute target from script, batch processing, termination and end of test, etc.

As shown in FIG. 2 , it is a schematic structural diagram of Embodiment 2 of the software testing system of the present invention.

The control machine specifically includes an analysis module, a test instance module, a data transmission module, a matching service client, a testing machine service client and a storage library. The parsing module is used to parse the main script and build a test instance module. The test instance module is used to execute the test process, call the matching machine to select the test machine, transmit the test data to the test machine and install the test environment on the test machine, send instructions to the test machine according to the test scenario, record the test results, and collect test data. The data transmission module is used to transmit test data through grid data service equipment. The matching service client is used to communicate with the matching machine, and call the matching machine to select the testing machine. The test machine service client is used to communicate with the test machine, transmit test data and instructions to the test machine, and receive the execution results, test results and test records returned by the test machine. Repositories are used to store test results.

The matching machine specifically includes an information collection module, an explanation module and a matching service module. The information collection module is used to obtain the attribute information of the test machine from the information service equipment of the grid; the interpretation module is used to judge whether the test machine meets the test requirements; the matching service module is used to communicate with the control machine, receive the matching request sent by the control machine, and send The control machine returns matching results. When an expression is used to describe whether the test machine meets the test requirements, the interpretation module needs to verify whether the syntax of the expression is correct, and judge whether the test machine meets the test requirements through the operation result of the expression.

The test machine specifically includes a test service module, a data transmission module, a test execution module and an information registration module. The test service module is used to communicate with the control machine, receive instructions and return execution results; the data transmission module is used to transmit test data through the data service equipment of the grid; the test execution module is used to perform corresponding operations according to instructions; the information registration module is used to Register the attribute information to the information service device of the grid, so that the matching machine can obtain the information of the testing machine. The registration of the attribute information of the test machine can adopt the method of regular registration, and the test machine regularly registers its own attribute information with the information service equipment of the grid, or other methods can be used, for example, the test machine sends information to the information service device every time its own attribute information changes. Service device registration.

In this embodiment, the grid is used to test the software, the data service equipment of the grid is used for data transmission, and the information service equipment of the grid is used to obtain the attribute information and function information of the testing machine; the high degree of parallelism of the grid can accelerate the execution of the test , improve test efficiency; the grid is a highly heterogeneous environment, and its rich resources can provide a variety of test environments for testing, reduce the cost of acquisition and maintenance of test environments, and these environments are closer to the real operating environment of the software; The synergy of the grid allows machines distributed in various places to jointly complete a test task and improve resource utilization.

The operation of the software under test may depend on other software, typically a database. When testing, it is not necessary to transfer a database system to the testing machine for installation every time it is tested. The testing machine can maintain some commonly used software, such as different databases. If a certain test needs these software, it can be obtained and installed locally. In order to enable other devices to know what software the test machine has locally, the test machine can publish its own software list to the information service device of the grid for other devices to query.

Each testing machine can provide different testing services to the outside world. For example, some can provide performance testing, and some can perform stress testing. The testing machine manages such services and releases its own functions to the outside world. This information can be published to the grid. on the information service device.

As shown in FIG. 3 , it is a schematic structural diagram of Embodiment 3 of the software testing system of the present invention. The difference between this embodiment and the second embodiment is that the test machine further adds a dependency library management module and a test function management module. The dependency library management module is used to maintain the local software of the testing machine and publish the software list to the information service device of the grid for other devices to query. The test function management module is used to manage such services and publish function information to the information service equipment of the grid. Therefore, for the software commonly used in testing, it is not necessary to retransmit and install it every time it is used, thereby reducing network transmission and speeding up the testing process. Other devices in the grid can also know what kind of testing services the testing machine provides. Select the testing machine more efficiently for more effective testing.

Since the tested software is likely to have defects, these defects may endanger the safety of the test machine during the test, for example, delete the test machine system files, write a large amount of data to the hard disk, etc. Therefore, it is necessary to ensure that the test machine is safe during the test safety.

As shown in FIG. 4 , it is a schematic structural diagram of Embodiment 4 of the software testing system of the present invention. The difference between this embodiment and the third embodiment is that the testing machine further adds a security module. This module is used to ensure the safety of the test machine during the test and avoid things that threaten the safety of the test machine.

As shown in Figure 5, it is a flow chart of the first embodiment of the software testing method of the present invention, including the following steps:

Step 101, analyze the main script, and set up a test instance; the test data description, test machine configuration description and test scene are recorded in the test script;

Step 102, acquiring test data according to the test data description;

Step 103, matching the testing machine according to the testing machine configuration description;

Step 104, transmit the test data according to the matching result, install the test environment, and load the slave script;

Step 105, execute the test scenario, collect test records, and uninstall the test environment.

This embodiment realizes the automation of software testing, and multiple testing machines cooperate to complete the testing, which can effectively improve the testing efficiency; since the testing machines are dynamically selected, it is possible to test software in various testing environments.

When the matching fails, a retry mechanism can be added. At the same time, a maximum value of the number of retries can be set, and if a suitable testing machine cannot be found after the number of retries reaches the maximum value, the test process will be terminated. A certain time interval can be set between each retry. Since grid resources are dynamic, the retry mechanism will effectively improve the matching success rate.

When the control machine executes the test scene, it obtains an unexecuted scene from the test scene list, sends an instruction to execute the running target to the test machine; sends an instruction to execute the running result target to the test machine according to the execution result returned by the test machine.

As shown in Figure 6, it is the execution flow chart of the control machine of the second embodiment of the software testing method of the present invention, including the following steps:

Step 201, receiving the main script submitted by the user, parsing the main script, and establishing a test instance in memory;

Step 202, obtain the test data from various data sources (such as CVS, FTP) to the local, and classify and package according to the test requirements; the test data required by different test machines is often the same, and also has its own uniqueness, therefore, you can Classify and package test data to reduce the amount of data transfer;

Step 203, sending a matching request to the matching machine according to the test requirements;

Step 204, receive the matching result, judge whether the matching is successful, if successful, execute step 207, otherwise, execute step 205;

Step 205, judging whether the number of matches reaches the maximum value, if so, end; otherwise, execute step 206;

Step 206, wait for a predetermined time, and execute step 203;

Step 207, transfer the packaged test data to the testing machine through the data service provided by the data service equipment of the grid, and install a test environment on the testing machine (comprising some pre-set environment variables, directories and test required configuration and installation of dependent software, etc.);

Step 208, execute the first "run" target of the unexecuted test scenario in the main script, and send an instruction to execute the "run" target to the testing machine;

Step 209, receiving the execution result, judging whether it is passed, if passed, then execute step 210, otherwise, execute step 211;

Step 210, execute the "success" goal, and send an instruction to execute the "success" goal to the testing machine;

Step 211, judging whether the execution result of the "running" object is a failure, if so, execute step 212, otherwise, it means that the execution result is abnormal, and execute step 213;

Step 212, execute the "failure" target, and send an instruction to execute the "failure" target to the testing machine;

Step 213, execute the "wrong" target, and send an instruction to execute the "wrong" target to the testing machine;

Step 214, receiving the execution result returned by the testing machine and registering the test result;

Step 215, determine whether all test scenarios have been executed, if so, perform step 216, otherwise, perform step 208;

Step 216, sending completion instructions to all testing machines to collect test records of the testing machines and uninstall the testing environment of the testing machines;

Step 217: Receive the test records generated by the test process sent by each test machine. The test records mainly include the output data and logs generated by the tested software during operation. The specific content of these records can be specified in the main script.

As shown in Figure 7, it is the execution flowchart of the testing machine of the second embodiment of the software testing method of the present invention, including the following steps:

Step 301, receiving test data, and unpacking and placing the test data into a suitable directory;

Step 302, installing the test environment;

Step 303, waiting to receive the instruction sent by the control machine;

Step 304, judging whether the instruction is a completion instruction, if so, execute step 310, otherwise, execute step 305;

Step 305, determine whether the instruction is a termination instruction, if so, execute step 311, otherwise, execute step 306;

Step 306, execute the instruction;

Step 307, whether abnormality occurs, if so, execute step 308, otherwise, execute step 309;

Step 308, report abnormality to the control machine, and execute step 303;

Step 309, send the command execution result to the control machine, and execute step 303;

Step 310, sending test records to the control machine;

Step 311 , uninstall the test environment, and clean up the environment variables, directories and dependent software used by the test.

The technical scheme of the present invention uses the grid to test the software, and the high parallelism of the grid can accelerate the execution of the test and improve the test efficiency; the grid is a highly heterogeneous environment, and the richness of its resources can provide a variety of tests for the test. The test environment reduces the acquisition and maintenance cost of the test environment, and these environments are closer to the real operating environment of the software; the synergy of the grid allows machines distributed in various places to jointly complete a test task and improve resource utilization. In addition, the grid application program and the grid middleware can also use the grid itself to improve their own quality, reduce the test cost and time, and realize the use of the grid to test the grid.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them; although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the present invention can still be Modifications to the specific implementation of the invention or equivalent replacement of some technical features; without departing from the spirit of the technical solution of the present invention, should be included in the scope of the technical solution claimed in the present invention.

Claims (5)

1, a kind of method for testing software is characterized in that, said method comprising the steps of:

Resolve test script, set up test case; Record test data in the described test script and describe, the test machine configuration is described and test scene;

Test data is obtained in description according to test data;

Configuration is described and is carried out the test machine coupling according to test machine;

If the match is successful, the data, services transmitting test data that the data service unit by grid provides, the installation testing environment if it fails to match, judges whether matching times reaches maximal value, if, end; Otherwise, receive matching result after the wait schedule time, judge whether success;

One of obtaining in the test scene tabulation of controller does not carry out scene, sends the instruction of carrying out operational objective to test machine, and sends the instruction of carrying out the operation result target to test machine according to the operation result that test machine returns; Receive execution result and register test result; Judge whether that all test scenes all execute, if not, carry out operational objective; When each scene is finished, controller is sent completely instruction to test machine; The acceptance test record;

Test machine is carried out and is operated and return execution result accordingly according to the instruction that receives; If described instruction for finishing instruction, then sends test record to controller, the unloading test environment; If described instruction is a command for stopping, then unload test environment; If described instruction then sends execution result to controller for the instruction of carrying out operational objective or the instruction of carrying out the operation result target after execution; If occur unusual, then unusual in carrying out to the controller report.

2, method for testing software according to claim 1 is characterized in that, described method also comprises: test machine is to mesh services information equipment registration attribute information, and issuing function information and local software are tabulated.

3, a kind of software testing system is characterized in that, described system comprises:

Controller is used to resolve test script, sets up test case, call MM and select test machine, test data is transferred to test machine and on test machine the installation testing environment, send instruction according to test scene to test machine, logging test results is collected the test record on each test machine;

MM is used for the test machine of matching test needs;

Test machine is used for the installation testing environment, and carries out corresponding operation according to the instruction that controller sends;

Described controller comprises:

Parsing module is used to resolve test script, sets up the test case module;

Data transmission module is used for by grid data service equipment transmitting test data;

The match service client is used for communicating by letter with MM, calls MM and selects test machine;

The test machine service client is used for communicating by letter with test machine, and test data and instruction are transferred to execution result, test result and the test record that test machine and acceptance test machine return;

Thesaurus is used for store test results and test record;

Described MM comprises:

The information gathering module is used for obtaining from the information service equipment of grid the attribute information of test machine;

Explanation module is used to judge whether test machine satisfies testing requirement;

The match service module is used for communicating by letter with controller, receives the matching request that controller sends, and returns matching result to controller;

Described test machine comprises:

Data transmission module is used for the data service unit transmitting test data by grid;

The testing service module is used for communicating by letter with controller, receives instruction and returns execution result;

Testing execution module is used for carrying out corresponding operation according to instruction;

The information Registering modules is used for attribute information is registered to the information service equipment of grid.

4, software testing system according to claim 3 is characterized in that, described test machine further comprises:

Rely on database management module, be used for the software of maintenance test machine local and software matrix be published to the information service equipment of grid;

The test function administration module is used for the test function that the management testing machine provides, and test function information is published to the information service equipment of grid.

5, software testing system according to claim 3 is characterized in that, described test machine further comprises: security module is used for ensureing the security of test process test machine.

Images