CN119065963A

CN119065963A - A testing method and computing device

Info

Publication number: CN119065963A
Application number: CN202411035882.5A
Authority: CN
Inventors: 徐军
Original assignee: XFusion Digital Technologies Co Ltd
Current assignee: XFusion Digital Technologies Co Ltd
Priority date: 2024-07-30
Filing date: 2024-07-30
Publication date: 2024-12-03

Abstract

The embodiment of the application provides a test method and computing equipment. The method comprises the steps of obtaining a test task, wherein the test task comprises a plurality of test cases, obtaining test environment requirements of each test case, determining a plurality of first test case sets according to the test environment requirements, determining a mutual exclusion array and at least one containing array according to the plurality of first test case sets, creating a corresponding test environment for each first test case set in the mutual exclusion array, distributing each first test case set to the corresponding test environment, creating a corresponding test environment for each containing array, distributing all first test case sets in each containing array to the corresponding test environment, and executing the test task. According to the scheme provided by the application, the number of the test environments can be accurately provided under the condition of diversity of the test environments, so that the waste of the test environment resources is reduced, and the test efficiency is improved.

Description

Test method and computing device

Technical Field

The present application relates to the field of server technologies, and in particular, to a testing method and a computing device.

Background

The environmental preparation for operating system testing is currently complex relative to the environmental deployment of application software, as the operating system involves more system-level configuration and setup. Typically, the required operating system test environment needs to be manually prepared before test cases for the operating system are executed. This way of manually preparing the environments is time consuming and does not accurately prepare the required number of environments. If the number of environments is too small, the test time is too long, and if the environments are prepared excessively, resources are wasted.

Disclosure of Invention

The embodiment of the application provides a testing method and computing equipment, which can provide proper quantity of testing environments so as to reduce waste of testing environment resources and improve testing efficiency.

For this reason, the following technical scheme is provided in the embodiment of the application:

According to the first aspect, the embodiment of the application provides a testing method, which comprises the steps of obtaining a testing task, obtaining testing environment requirements of each testing case, determining a plurality of first testing case sets according to the testing environment requirement information, wherein the testing environment requirements of the testing cases contained in the first testing case sets are the same, determining a mutual exclusion array and at least one containing array according to the first testing case sets, wherein the mutual exclusion array comprises at least one first testing case set, different first testing case sets do not have containing relations, the containing array comprises a plurality of first testing case sets, the containing relation exists among the different first testing case sets, a corresponding testing environment is created for each first testing case set in the mutual exclusion array, and each first testing case set is distributed to the corresponding testing environment, the testing environment requirements of the distributed first testing case sets are met, the mutual exclusion array comprises at least one first testing case set, the containing relation among the different first testing case sets is not contained among the first testing case sets, the containing case sets are contained in the corresponding testing environment is created for each first testing case set, and each first testing case set is distributed to the corresponding testing environment, and the first testing case set is distributed to the corresponding testing environment is distributed to the testing environment.

In this embodiment, first, a plurality of test cases are acquired according to test requirements of an operating system, and the test cases require a specific test environment when executed. The test cases required by the same test environment are grouped, so that the utilization of the environment is optimized preliminarily, and the resource waste is avoided. The method and the device are used for optimizing different groups of test cases, for example, calculating the needed set inclusion relation of the disk information for determining the overlapping relation of the disk information, so as to accurately determine the number of test environments to be configured, and effectively configure the test environments. Therefore, the test cases are reasonably grouped and sequenced according to the test environment requirements of the test cases, the first test case set is optimized, and the number of accurate test environments can be obtained after the required test environments of the test cases of the operating system are obtained, so that the waste of test environment resources is reduced, and the test efficiency is improved.

As one implementation, the test environment requirements include processor, memory, and disk information.

In this embodiment, in the test environment requirements, configuration of hardware resources (such as processors, memory, and disks) is typically involved to ensure that the test environment effectively simulates the operating environment and supports the required test activities. For example, by configuring a high performance processor, a high capacity memory, and a high speed disk, the test environment may better simulate the performance level in the system operating environment, thereby more accurately evaluating the performance of the system in actual use.

As one implementation manner, the determining the mutex array and the at least one inclusion array according to the plurality of first test case sets comprises determining the mutex array and the at least one inclusion array based on disk information in the test environment required by the first test cases.

The method comprises the steps of determining a mutual exclusion array and at least one inclusion array based on disk information in a test environment required by first test cases, traversing each first test case set, taking the first test case set and the first test case set with the inclusion relationship as an inclusion array under the condition that the inclusion relationship between the first test case set and the disk number and the disk size required by any one of the first test case sets exists, and putting the first test case set into the mutual exclusion array under the condition that the inclusion relationship between the first test case set and the disk number and the disk size required by any one of the first test case sets does not exist.

In this embodiment, in a test environment where multiple test cases are tested, disk information is often a relatively stable and important factor in the configuration of the test environment, affecting the storage and data processing capabilities of the test cases. Therefore, the test case set which can share the same disk resource can be more easily divided by screening with the disk as the main part, so that more stable and reliable test environment division can be provided, the test case can be ensured to operate under the similar test environment condition, and the test efficiency and the repeatability are improved.

The method further comprises the step of distributing the first test case set with the disk information of 0 to the mutex array and the first test case set comprising the array based on the processor and/or the memory configuration information in the test environment requirement of the first test case set and the processor and/or the memory configuration information of the test environment requirement of the first test case set comprising the mutex array under the condition that the disk information in the test environment of the first test case set test case is 0.

In this embodiment, under the condition that the number of disks and the size of the disks in the test environment are both 0, the first test case set is allocated to other first test case sets based on the number of processor cores and/or the memory configuration information in the test environment, so as to ensure that the test cases after subsequent merging can obtain computing resources (processors and memories) meeting the requirements of the test cases, avoid resource waste and shortage, ensure that all the test cases can operate efficiently, and shorten the overall test time.

As an implementation manner, the creating a corresponding test environment for each first test case set in the mutex array includes creating a processor core number of the test environment according to a processor core number in a test environment requirement of the first test case set, creating a memory size of the test environment according to a memory size in the test environment requirement of the first test case set, and creating disk information of the test environment according to disk information in the test environment requirement of the first test case set.

In this embodiment, the test environment may configure the number of processor cores, the memory size, and the disk information according to the actual test environment requirements of each first test case set, so as to maximize performance, thereby ensuring efficient execution of the test, and avoiding the problem of test execution caused by insufficient hardware resources.

As an implementation manner, the creating a corresponding test environment for each of the containing arrays includes creating disk information of the corresponding test environment according to disk information that is needed most by a first test case set in the containing array, creating processor cores of the corresponding test environment according to processor cores that are needed most by the first test case set in the containing array, and creating memory size of the corresponding test environment according to memory size that is needed most by the first test case set in the containing array.

In this embodiment, the test environment is created by the maximum number of processor cores, the memory size, and the disk requirements in each first test case set in the array, ensuring that the test environment is sufficient to meet the maximum requirements of each test case. Therefore, only one test environment is needed to be created for each containing array, no extra redundant environment or excessive resource allocation is needed, and cost and resources are saved, so that resource waste and performance bottlenecks are avoided.

The method comprises the steps of establishing a corresponding test environment for each containing array, determining whether the total number of test cases of a plurality of first test case sets in each containing array is smaller than or equal to a preset number, establishing a test environment for each containing array when the total number of test cases of the plurality of first test case sets in each containing array is smaller than or equal to the preset number, and distributing all the first test case sets in each containing array to the corresponding test environment, and establishing a test environment corresponding to the ratio for the first test case sets in each containing array according to the ratio of the total number/the preset number when the total number of test cases of the plurality of first test case sets in each containing array is larger than the preset number.

In this embodiment, the resource allocation can be effectively optimized by calculating the comparison of the total number of test cases with the preset number. When the number of the test cases exceeds the preset number, the number of the test environments can be minimized, resources are saved, and when the number of the test cases exceeds the preset number, the test environments are dynamically increased according to the ratio, so that the reasonable load of each test environment is ensured. In addition, the number and the configuration of the test environments are dynamically adjusted according to actual conditions, the test requirements of different scales and complexity can be flexibly met, and the flexibility can improve the efficiency and the reliability of test execution. Finally, by precisely calculating and distributing the test resources, resource waste and over-configuration can be avoided, thereby ensuring efficient execution and accuracy of the test process. Each test environment can run the distributed test cases under proper conditions, so that the problems of resource contention and performance bottleneck are avoided. Meanwhile, the scheme can realize dynamic management and configuration of the test environment by means of an automatic tool, reduces complexity and possibility of errors of manual adjustment, and improves overall management efficiency and quality control.

As an implementation mode, the method further comprises the steps of releasing the test environment when the test case is successfully executed, reserving the test environment when the test case is failed to be executed, and releasing the test environment when the reserved time length reaches a preset time length.

In this embodiment, when the test case fails to execute, the test environment is preserved for a predetermined time. This is done to facilitate subsequent debugging analysis of the cause of failure while avoiding the additional overhead and time consumption that is added by frequently creating and destroying the environment. Conversely, when the test case is successfully executed, the test environment used is released immediately. In this way, resources can be reassigned to other test tasks to be performed to maximize resource utilization, ensuring that resources are not wasted by not freeing an environment that is no longer needed.

As one implementation, the test environment includes one or more of a network card, GPU, or SATA configuration information.

In this embodiment, in combination with the network card, GPU and SATA configurations, the test environment can simulate a variety of usage scenarios, thereby more fully assessing the performance and stability of the system.

In a second aspect, an embodiment of the present application further provides a testing device, including:

The acquisition module is used for acquiring a test task, wherein the test task comprises a plurality of test cases and acquires the test environment requirement of each test case;

The grouping module is used for determining a plurality of first test case sets according to the test environment demand information, wherein the test environment demands of all the test cases contained in one first test case set are the same;

The construction module is used for determining a result array mutex array and at least one inclusion array according to a plurality of first test case sets; the mutex array comprises at least one first test case set, and no relation exists between different first test case sets; the containing array comprises a plurality of first test case sets, and containing relations exist among different first test case sets; the construction module is also used for creating a corresponding test environment for each first test case set in the mutex array, and distributing each first test case set to the corresponding test environment, wherein each test environment meets the test environment requirements of the distributed first test case set;

And the test module is used for executing the test task.

As one implementation, the construction module determines a mutually exclusive array and at least one inclusion array based on disk information in the test environment required by the first test case.

The construction module is further used for traversing each first test case set, taking the first test case set and the first test case set with the inclusion relationship as an inclusion array under the condition that the inclusion relationship exists between the first test case set and the number of disks required by any one of the first test case sets and the size of the disks, and putting the first test case set into the exclusive array under the condition that the inclusion relationship does not exist between the first test case set and the number of disks required by any one of the first test case sets and the size of the disks.

As an implementation manner, the construction module is further configured to create the number of processor cores of the test environment according to the number of processor cores in the test environment requirement of the first test case set, create the memory size of the test environment according to the memory size in the test environment requirement of the first test case set, and create the disk information of the test environment according to the disk information in the test environment requirement of the first test case set.

As an implementation manner, the construction module is further configured to create disk information corresponding to the test environment according to the disk information that is needed most by the first test case set in the array, create processor cores corresponding to the test environment according to the processor cores that are needed most by the first test case in the array, and create memory size corresponding to the test environment according to the memory size that is needed most by the first test case in the array.

The construction module is further configured to determine whether a total number of test cases of the plurality of first test case sets in each containing array is less than or equal to a preset number, create a test environment for each containing array when the total number of test cases of the plurality of first test case sets in each containing array is less than or equal to the preset number, and allocate all the first test case sets in each containing array to the corresponding test environment, and create a test environment corresponding to the ratio for the first test case sets in each containing array according to a ratio of the total number/the preset number when the total number of test cases of the plurality of first test case sets in each containing array is greater than the preset number.

The test device further comprises a monitoring module, a test environment releasing module and a control module, wherein the monitoring module is used for monitoring an execution result of the test case in real time, releasing the test environment when the test case is successfully executed, reserving the test environment when the test case is failed to be executed, and releasing the test environment when the reserved time length reaches a preset time length.

In a third aspect, embodiments of the present application also provide a computing device comprising a memory and a processor;

the memory is coupled to the processor;

a memory for storing a program;

A processor for invoking a memory-stored program to cause a computing device to perform the method as referred to in the first aspect and its possible implementations.

In a fourth aspect, embodiments of the present application also provide a computer-readable storage medium storing computer instructions that, when executed by a computing device, cause the computing device to perform the method of the first aspect and its possible implementation manner.

In a fifth aspect, embodiments of the present application also provide a computer program product comprising computer instructions which, when executed by a computing device, cause the computing device to perform the method of the first aspect and possible implementations thereof.

Drawings

Fig. 1 shows an application scenario schematic diagram of a test method provided by an embodiment of the present application;

FIG. 2 is a schematic flow chart of a test method according to an embodiment of the present application;

FIG. 3 illustrates a flow chart for partitioning a first plurality of test case sets into mutually exclusive arrays and containing arrays;

FIG. 4 shows a flow diagram of creating a corresponding test environment for each containing array;

FIG. 5 is a flow chart of another testing method according to an embodiment of the present application;

FIG. 6 is a flow chart of the test method according to embodiment 1 of the present application;

FIG. 7 is a schematic structural diagram of a testing device according to an embodiment of the present application;

FIG. 8 is a schematic diagram of another computing device provided in an embodiment of the application.

Detailed Description

The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application.

In describing embodiments of the present application, words such as "exemplary," "such as" or "for example" are used to mean serving as examples, illustrations or explanations. Any embodiment or design described herein as "exemplary," "such as" or "for example" is not necessarily to be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary," "such as" or "for example," etc., is intended to present related concepts in a concrete fashion.

In the description of the embodiment of the present application, the term "and/or" is merely an association relationship describing the association object, and indicates that three relationships may exist, for example, a and/or B, and may indicate that a exists alone, B exists alone, and both a and B exist. In addition, unless otherwise indicated, the term "plurality" means two or more. For example, a plurality of systems means two or more systems, and a plurality of terminals means two or more terminals.

Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating an indicated technical feature. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise.

In the description of embodiments of the application reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is to be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments and can be combined with each other without conflict.

In the description of the embodiments of the present application, the terms "first\second\third, etc." or module a, module B, module C, etc. are used merely to distinguish similar objects and do not represent a particular ordering for the objects, it being understood that particular orders or precedence may be interchanged as allowed so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

In the description of the embodiments of the present application, reference numerals indicating steps, such as S31, S32. It is not meant to necessarily be performed in this step, and the order of the steps may be interchanged, or performed simultaneously, where permitted.

Related terms related to embodiments of the present application:

Test environment-refers to a description of the operating system and hardware environment on which the test is running, as well as any other software that interacts with the operating system under test, including configurations in terms of hardware, software, and networks. This environment is typically specially configured so that a developer, tester, or automated test tool can perform various tests therein, including unit testing, integrated testing, system testing, and the like. The test environment is stable and controllable, so that a tester can spend less time to complete the execution of the test case, additional time is not required to be spent for the maintenance of the test case and the test process, and each submitted defect can be ensured to be accurately reproduced at any time.

The number of test environments refers to the number of different test environments that are independent of each other and are required when operating system testing is performed. Typically, the number of test environments depends on the parallel test requirements, and if multiple independent test tasks (such as parallel testing) need to be run simultaneously, multiple test environments are required. The resources (e.g., CPU, memory, storage space) required by each test environment and their availability can impact the number of test environments. By increasing the number of test environments, the efficiency and speed of the test can be increased, thereby obtaining test results faster. The number of test environments is determined based on the size, nature, and resource requirements of the test tasks, and is intended to support efficient, effective, and reliable operating system test activities.

Test cases the test cases of the operating system test refer to specific test scenes or test steps for verifying and evaluating the functions, performances and stability of the operating system, and specifically refer to description of test tasks performed on a specific operating system, so that test schemes, methods, technologies and strategies are embodied. When testing an operating system with test cases, each test case should specifically describe one or more expected behaviors, as well as the input and execution conditions required to trigger those behaviors. Briefly, a test case is a set of inputs, execution conditions, expected results, and execution steps used to verify a particular function, feature, or performance during an operating system test. Test cases are intended to verify that an operating system is working as intended and can be used to discover potential defects or problems.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of the application only and is not intended to be limiting of the application.

With the continuous progress of computer technology, the operating system is more and more complex, the related operating system software and functions are more and more complex, the probability of occurrence of faults of the operating system is increased, and the faults are found and solved more difficult, so that the operating system needs to be tested by adopting test cases to verify the functions, compatibility and performance of the operating system.

Typically, the test cases for operating system testing require manual preparation of the required test environment prior to execution. For example, currently, the main flow in the industry is to evaluate the number of test environments required by test cases under the name of a tester, then manually deploy and create an operating system test environment, then deploy a test framework and execute a test, and manually delete the operating system and the test environment after the test is finished. Manually preparing the test environment requires the tester to operate one by one, including installing an operating system, configuring a network, installing a software package, adjusting system settings, and the like. This approach is time consuming and prone to error, particularly in a large scale test environment, manual management becomes impractical. Moreover, manually preparing the environment often does not accurately predict the number of environments required for testing. If the number of environments is insufficient, the test resources are possibly insufficient, the test time is prolonged, and if the number of environments is excessive, the resources are wasted.

Based on the above, the embodiment of the application provides a testing method and computing equipment. In the method, the accurate test environment quantity required by the test case is calculated by acquiring the requirements of the test case on the test environment during self execution recorded in the test case and arranging and combining hardware resource information (such as a processor (central processing unit, cpu), a memory (memory), a disk (disk), a network card (netcard) and the like) of the test environment required by the test case, and the operating system test environment is automatically deployed and created according to the calculated test environment quantity. Therefore, the number of the test environments can be accurately provided under the condition of diversity of the test environments, so that the resource waste of the test environments is reduced, and meanwhile, the test efficiency is better improved.

In order to make the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be described in detail below with reference to the accompanying drawings. The specific embodiments illustrated may be combined with one another and the same or similar concepts or processes may not be described in detail in some embodiments.

Fig. 1 shows an application scenario schematic diagram of a test method provided by an embodiment of the present application. As shown in fig. 1, the application scenario includes a management device 101 and a plurality of test devices 101, and the management device 101 and the test devices 101 are different electronic devices, but have the same architecture type.

It should be noted that, the management device 101 is connected to the plurality of test devices 101, where a communication connection is established between the management device 101 and the test devices 101 through a wired network (wireless network) or a wireless network (wireless network) for data interaction. For example, the network may be a local area network (local area networks, LAN) or a wide area network (wide area networks, WAN) (e.g., the internet). The communication connection manner between the management device 101 and the test device 101 is not limited in the embodiment of the present application.

For example, the management device 101 may be a server or a terminal device, where the management device 101 is configured to provide a plurality of test cases based on a test task, and the test cases carry the required test environment requirements of the test cases. The management device 101 may obtain a test environment requirement of each test case, and determine a plurality of first test case sets according to the test environment requirement information, where the test environment requirement of each test case included in any one of the first test case sets is the same.

The management device 101 is further configured to determine a mutex array and a containing array according to the plurality of first test case sets, where the mutex array includes at least one first test case set, and no containing relationship exists between different first test case sets in the mutex array, and the containing array includes the plurality of first test case sets, and contains a containing relationship between different first test case sets in the containing array. The management device 101 respectively downloads the mutually exclusive array and the first test case set in the array into different test devices 101 according to different test environment requirements of the test cases.

The test equipment 101 related in the embodiment of the application is used for providing the test environment required by the first test case set, and different test equipment 101 is used for providing different test environments so as to meet the requirement that the test cases in different first test case sets execute test tasks. For example, the test equipment 101 may be a hardware server or a virtualized server, e.g., a test environment in the test equipment 101 may be configured by a virtual machine executing on a hardware server that includes one or more other virtual machines.

The above is an introduction to an application scenario for executing the test method involved in the present solution. Next, based on the above, a flow of the test method provided by the embodiment of the present application will be described.

Fig. 2 shows a flow chart of a test method according to an embodiment of the application. The scheduling method is adapted to be executed in a computing device, such as the aforementioned management device 101. As shown in fig. 2, the test method includes:

S101, acquiring a test task, wherein the test task comprises a plurality of test cases.

In this step, a plurality of test cases to be executed may be obtained based on a test task of the operating system, and it is understood that in the test task based on the operating system, a "test task" generally refers to an overall test process or plan, which includes execution and management of the plurality of test cases. Each test task may be a batch job or a set of related test operations intended to evaluate the performance, stability, or functionality of the operating system under specific conditions.

And the test case is a basic execution unit of the test task. Each test case describes a specific test behavior or function point, and typically contains information such as input data, expected output, execution steps, etc. It should be noted that, the data of the test case, the test result, and the like may be stored by a database, such as a relational database (e.g., mySQL, postgreSQL) or a NoSQL database (e.g., mongoDB).

S102, obtaining the test environment requirement of each test case.

It should be noted that the test case carries the required information of the test environment, where the test environment includes the number of processor cores, the memory size, the disk information and other hardware environment resource information that need to be configured when the test case is executed.

Illustratively, each test case carries information about the test environment required for execution. Such information may include operating system configuration such as operating system type (Windows, linux, etc., linux operating system in this embodiment), version number, hardware configuration such as CPU core number, memory size, disk information, network card settings, etc., where the disk information describes the number of disks and disk space required during test execution, read/write speed, etc. This information is critical to the execution of certain test cases, such as those requiring extensive data storage or IO operations.

Specifically, when the test cases are compiled, a developer or a tester can record the test environment information required by each test case, including an operating system, a software version, configuration parameters, network settings and the like. In other words, when writing test cases, the results of using fixed data for environmental appeal of the operating system are recorded, and such information can be embedded in the test cases in a binary manner (language recognizable by the computing device) in the form of notes, documents, or specific indicia (e.g., keywords) for later use. The embodiment can be used for obtaining the hardware resource configuration information in the test environment by identifying the keywords of each test case, such as the CPU core number, the memory size, the disk information, the network card setting and the like. Of course, in some possible implementations, the hardware resource configuration information required by the test environment may further include:

the high quality network card (netcard) configuration ensures stability and speed of network communications, which is critical to testing distributed systems, network services, or applications requiring high bandwidth.

GPU (graphics processing unit ) information GPU model, video memory size, and driver version may be necessary information for test cases that require graphics processing or computationally intensive tasks.

USB and other interface information if testing requires the use of a specific USB device or other interface, such as SATA (SERIAL ADVANCED technology attachment ), thunderbolt, etc., the relevant interface type and version information needs to be recorded.

Thus, S102 may obtain the test environment requirement information when the test case is executed based on the test environment to which the test case is bound.

S103, determining a plurality of first test case sets according to the test environment demand information of the test cases, wherein the test environment demands of all the test cases contained in one first test case set are the same.

In the step, a plurality of test cases are arranged and combined, and the test cases with the same test environment are divided into a group. The method is used for preliminarily integrating the test environments required by the test cases of the operating system, and by means of permutation and combination calculation of the test environment information, how many test cases of the operating system can be deployed in a given test environment can be determined. Specifically, the processor, the memory, the disk and the network card configuration information required in the test environment can be grouped, and the test cases with the same requirement of the test environment are grouped into the same group, so that the quantity of the operating system test required for the test environment is preliminarily determined. And then sequencing the test cases in the same group to obtain the number of the test cases in the same group.

For example, if there are 25 test cases, they can be grouped into groups, with the test cases in each group having the same test environment requirements, see in particular Table 1.

TABLE 1 test case grouping with the same test Environment and examples of the required cpu, mem and disk information in the test environment

As can be seen from Table 1, the 25 test cases are divided into 6 groups of first test case sets according to different test environment requirement information, wherein the test environment information in Table 1 comprises a processor cpu, a memory mem and a disk. Specifically, in the cpu information, each row represents a cpu configuration of a test environment, where the numbers 2,4, and 8 represent a cpu core number. For example, a group 1 test environment has 2 cpu cores, a group 2 test environment has 4 cpu cores, and so on. Memory information, each row represents the memory configuration of a test environment, and each number represents the memory size in GB. For example, the first test environment has 4GB of memory, the second test environment has 4GB of memory, and so on. Disk information, each row represents the disk configuration of a test environment. The disc information is separated by commas, and each number represents the size of one disc in GB. For example, the first column of test environments has one 50GB and one 30GB disk, the second column of test environments has one 30GB and one 10GB disk, and so on. And the test case information is that each row represents a test case in a test environment, the test cases are listed in sequence, and each row contains one or more test case names. For example, the first test environment contains two test cases testcase and testcase, the second test environment contains three test cases testcase, testcase, and testcase, and so on. The information shown in the table 1 can be used for configuring and managing the test environments to preliminarily determine the number of the test environments, so that the test cases can be executed and verified under the test environments required by the test cases.

Therefore, the test cases with the same requirements of the processor, the memory and the disk are classified into the same group, so that the test cases in the same group can share the same test environment, the number of the test environments to be configured is preliminarily determined, the waste of resources is reduced, and meanwhile, the test efficiency can be improved. In some embodiments, the test cases of the same group are ordered according to their priority, execution time, dependency, etc., to determine the execution order. Meanwhile, the number of test cases in each group is counted, so that resource allocation and test plan making are facilitated. Therefore, the test cases in the same test environment can share the same set of resource allocation, so that hardware resources (such as processors, memories and magnetic disks) can be managed and allocated more finely, and the resource utilization rate is improved. Moreover, by reasonably sequencing and grouping, the test cases of the same test environment are concentrated to be executed in the same batch, so that frequent switching of the test environment configuration is reduced, and time and resources are saved. For example, for the test cases in the same group, the test cases can be continuously executed, so that the preparation time before and after the execution of the single test case is reduced, the waiting time among the test cases is effectively reduced, and the test execution efficiency is integrally improved.

S104, determining a mutex array and at least one inclusion array according to the plurality of first test case sets, wherein the mutex array comprises at least one first test case set, no inclusion relation exists among different first test case sets in the mutex array, the inclusion array comprises the plurality of first test case sets, and the inclusion relation exists among different first test case sets in each inclusion array.

The step is mainly used for integrating the plurality of first test case sets preliminarily divided in the step S103 so as to obtain more accurate test environment quantity.

For example, in the case that the test environment of the test case in the third test case set includes the test environment of the test case in the fourth test case set, the third test case set and the fourth test case set are combined in the same inclusion array, where the third test case set and the fourth test case set are any one of the plurality of first test case sets. It should be noted that, the inclusion relationship in this embodiment means that the parameter of a certain test environment of one first test case set includes the parameter of a certain test environment of another first test case set. Referring to table 1, the above parameters may be disc information including the number of discs and the disc size, for example. For example, the disk requirement information in the test environment requirement of the fourth test case set is 10GB (column 6 in table 1), and the disk information in the test environment requirement of the third test case set is 2 disks and the sizes of the 2 disks are 50GB and 10GB (column 3 in table 1), then the 50GB and 10G disk information of the test environment requirement of the third test case set contains 10GB disk information in the test environment requirement of the fourth test case set, because the 50GB and 10GB contain 10GB. At this time, the third test case set and the fourth test case set are merged in the same containing array. Similarly, the disk information required in the first test case set in the 4 th column in table 1 is 50GB, 10GB and 20GB, and includes the disk information in the 3 rd column of 50GB, 10GB and the disk information in the 6 th column of 10GB, and then the first test case set in the 4 th column in table 1 is merged into the containing array where the 3 rd column and the 6 th column are merged. That is, the test environment requirements between different first test case sets in the inclusion array have an inclusion relationship.

In another case, if the test environment of the test cases in the third test case set does not include the test environment of the test cases in the fourth test case set, the third test case set is partitioned into a mutually exclusive array. The third test case set and the fourth test case set are any one of the first test case set. In other words, the test environment requirements required between any two first test case sets in the mutex array are mutually exclusive. For example, the disk information required for the first test case set in column 1 in table 1 is 50GB and 30GB, and the disk information required for the first test case set in column 2 is 30GB and 10GB. It can be seen that the disk information of the test environment information in the first test case set in the 1 st column and the second test case set in the 2 nd column in table 1 does not contain each other, and the first test case sets in the 1 st column and the second column are combined into the same mutex array. In this way, in this step, based on the number of disks and the size of disks in the test environment of the test cases in the third test case set, and the number of disks and the size of disks in the test environment of the test cases in the fourth test case set, a containment relationship between the third test case set and the fourth test case set may be determined, and whether a containment relationship exists between the third test case set and the fourth test case set determines the mutually exclusive array and at least one containment array.

In one possible implementation, sorting and filtering may be performed based on disk information, where the first test case set requiring different disk information is divided into a mutex array (not including a group with each other) and an inclusion array. The reason is that in the test environment of multiple test cases, disk information is often a relatively stable and important factor in the configuration of the test environment, affecting the storage and data processing capabilities of the test cases. In contrast, processor (CPU) and memory (mem) choices may be more affected by the computing requirements and memory occupation of test cases, these factors may vary greatly from test case to test case, screening based on processor or memory may result in a large combination of variations, and it is difficult to determine which test cases may be stably running in the same test environment. Therefore, the test case set which can share the same disk resource can be more easily divided by screening with the disk as the main part, so that more stable and reliable test environment division can be provided, the test case can be ensured to operate under the similar test environment condition, and the test efficiency and the repeatability are improved.

In some implementations, FIG. 3 illustrates a flow chart of partitioning a first plurality of test case sets into mutually exclusive arrays and containing arrays. As shown in fig. 3, the plurality of first test case sets may be partitioned by:

s1041, traversing each first test case set.

In this step, disk information in the test environment required by a first test case set is defined as an array, and exemplary disk information in the test environment includes the number of disks and the size of the disks. Referring specifically to table 1, exemplary disk information 50,10,20 in the 4 th set of test environments in table 1 may be used as an array, representing that three disks are required for the disk information, and the sizes of the disks are 50GB, 10GB, and 20GB, respectively. And taking all the arrays (for example, 6 arrays in table 1) as original arrays, taking the original arrays as a set of all the arrays, and traversing from the first test case set with the largest disk number, so that the subsequent set is convenient to comprise a calculation step.

In one possible implementation, the array length, which is typically the largest in number of disks, is longer, traversing from the array with the largest array length, because the larger the number of disks, the greater the likelihood that the content of the other arrays is contained, e.g., the 4 th set of disk information data (50,10,20) contains the 4 th set of disk information data (50, 10) and the 6 th set of disk information data (10). Therefore, the combination of the arrays can be completed by less judgment times, so that the traversal times are reduced as much as possible when the arrays are combined, and the algorithm efficiency is improved. For example, in this step, an array of 4 th set of disk information data (50,10,20) is first added to the mutex array.

S1042, judging the inclusion relation among the test environments required by the first test case sets. In one case, if there is an inclusion relationship between the test environments required for the plurality of first test case sets, S1043 is performed. In another case, if there is no inclusion relationship between the test environments required for the plurality of first test case sets, S1044 is performed. Specifically, the determination of the inclusion relationship of the disk information may be based on the number and size of disks required by the test environments in the different first test case sets. For example, the 4 th set of disc information data in table 1 is (50,10,20), and by comparing the other sets of disc information data in table 1, we can obtain that the 3 rd set of disc information data (50, 10) and the 6 th set of disc information data (10) are both contained in the 4 th set of data, and the 3 rd, 4 th and 6 th sets of disc information are called as having a containing relationship. In table 1, the 1 st set of disk information data (50, 30) and the 2 nd set of disk information (30, 10) have a common disk of 30GB, but the 1 st set of disk information also needs a 50GB disk and the 2 nd set of disk information also needs a 10GB disk, so that the two sets of disk information are mutually exclusive and have no inclusion relationship. Similarly, the 1 st and 2 nd disk information and the 3 rd, 4 th and 6 th disk information are mutually exclusive because of the need of 30GB disk.

S1043, under the condition that a first test case set and any test environment required by the first test case set have a containing relation, the first test case set and the first test case set with the containing relation are taken as a containing array.

Illustratively, referring to Table 1, a first set of test cases having groups 3, 4, 6 of containment relationships is partitioned into a containment array.

S1044, under the condition that a first test case set and any test environment required by the first test case set do not have an inclusion relation, putting the first test case set into a mutex array.

For example, referring to table 1, if the disk information in the test environments required by the first test case sets of the 1 st group and the 2 nd group and the first test case sets of other groups are not included, the first test cases of the 1 st group and the 2 nd group are put into a mutually exclusive array.

Therefore, by carrying out the set inclusion calculation on the disk information, the hierarchical relationship and the overlapping condition between the data can be clearly obtained, the inclusion relationship between the data can be clarified, and the disk information data in the original array is divided into the mutex array and the inclusion array. After the traversal is finished, all the arrays that are not completely contained by other arrays (i.e. have a disk that is not needed by other arrays) are contained in the mutually exclusive arrays, and the arrays in the contained arrays have a containing relationship (i.e. the disk information needed by one array is completely contained in another array). For example, for the data in Table 1, we can get a mutually exclusive array of (50, 30), (30, 10), containing arrays of (50,10,20), (50, 10), and (10). Thus, we obtain non-overlapping, independent data sets (mutually exclusive arrays), and then further processing of the test cases contained in the arrays is required to facilitate the subsequent determination of the precise number of test environments.

S105, creating a corresponding test environment for each first test case set in the mutex array, and distributing each first test case set to the corresponding test environment, wherein each test environment meets the test environment requirements of the distributed first test case set.

Optionally, in this step, according to the test environment requirements required by each first test case set in the mutex array, a physical machine and/or a virtual machine is adopted to construct a corresponding test environment. It can be understood that the memory configuration information of the created test environment is equal to the memory configuration information cpu of the first test case set, the memory configuration information is equal to the memory configuration information mem required by the first test case set, and the disk information parameter is equal to the disk information disk of the first test case set.

By way of example, physical machines are deployed on which a test environment may be built directly if sufficient physical server or computer resources are available. It should be noted that, according to the number of test environments, multiple test environments may be deployed in parallel on different physical machines, so as to support parallel execution of test cases.

Virtual machine deployment-if physical machine resources are limited or more flexible test environment management is required, virtualization techniques (e.g., VMware, virtualBox, KVM, etc.) can be used to build the test environment. Through virtualization technology, multiple virtual machine instances can be created on one computing device (physical machine), and a separate test environment can be built in each virtual machine. This allows for more efficient use of physical machine resources and facilitates management and maintenance of the test environment.

S106, creating a corresponding test environment for each containing array, and distributing all the first test case sets in each containing array to the corresponding test environment, wherein each test environment meets the test environment requirements of the distributed first test case sets.

In the step, the disk information disk of the created test environment is equal to the disk information disk which contains the largest first test case set in the array, the processor cpu is equal to the processor cpu which contains the largest first test case in the array, and the memory mem is equal to the memory mem which contains the largest first test case in the array. Thus, the created test environment can meet the test operation of the test cases in all the first test case sets in the array. For example, the test environment may be created using a physical machine and/or a virtual machine, as described in detail in S105 above.

For example, the test environment requirement information of the first test case set in the array divided in Table 1 is set 3 (corresponding to CPU is 4, mem is 2, disk is 50, 10), set 4 (corresponding to CPU is 8, mem is 8, disk is 50, 10, 20) and set 6 (corresponding to CPU is 4, mem is 8, disk is 10), the maximum disk information disk is 50, 10, 20, the maximum processor CPU is 8, the maximum mem is 8, and the test environment created corresponding to CPU is 8, mem is 8, disk is 50, 10, 20. In this way, the test environment created correspondingly can meet the machine resources required by the 3 rd, 4 th and 6 th groups in table 1.

In one possible implementation, FIG. 4 shows a flow diagram of creating a corresponding test environment for each containing array. As shown in fig. 4, in one possible implementation, creating a corresponding test environment for each containing array may be implemented by:

s1061, determining whether the total number of test cases of each of the plurality of first test case sets in the array is smaller than or equal to a preset number. Under the circumstance, the total number of test cases including the plurality of first test case sets in the array is less than or equal to the preset number, and S1062 is executed. In another case, the total number of test cases including the plurality of first test case sets in the array is greater than the preset number, and S1063 is executed.

It will be appreciated that the above-mentioned preset number indicates the number of test cases that can be executed in a group under the same test environment.

It should be noted that the preset number is set because running too many test samples in the same test environment affects the test efficiency to some extent. Therefore, the limitation (preset number) of the test environment use cases is added in the embodiment, and the preset number can be reasonably adjusted according to the test tasks under the condition of sufficient environment resources, so that the test execution efficiency can be better controlled.

Specifically, the preset number may be set based on available hardware resources, such as the number of available CPU cores, the memory size, and disk space. The execution of test cases requires the consumption of these resources, so the preset number needs to ensure that the maximum resource limit that the computing device can provide is not exceeded to avoid resource bottlenecks and performance problems.

The preset number can also be set according to the execution time and the execution frequency of the test cases. If the test cases need to be executed for a long time or need to be run frequently, the preset number may be adjusted according to these requirements to ensure that the system can complete the test in a reasonable time.

Sometimes testing needs to be performed in parallel in multiple environments to speed up overall test time. The preset number of settings may also take into account this parallelism requirement to fully utilize available test resources and maximize test efficiency.

In practical applications, the preset number may be determined through experiments and performance tests to ensure that the test cases can be stably and efficiently run under the given resources, for example, the preset number may be 1-100, which is not limited herein.

S1062, under the condition that the total number of test cases of the plurality of first test case sets in each containing array is smaller than or equal to the preset number, creating a test environment for each containing array, and distributing all the first test case sets in each containing array to the corresponding test environments.

In the step, the disk information disk of the created test environment is equal to the disk information disk which contains the largest first test case set in the array, the processor cpu is equal to the processor cpu which contains the largest first test case in the array, and the memory mem is equal to the memory mem which contains the largest first test case in the array. The test environment may be created using a physical machine and/or a virtual machine, as described in detail in S104 above.

Illustratively, in this embodiment, the preset number is 20. The total of three test cases in the 3rd group are testcase, respectively, 7-testcase. The number of test cases in the 6 th group is four, which are testcase to testcase respectively. And six test cases of the 5 th group are testcase, 10-testcase respectively. Thus, the total number of test cases in the first test case set of the three groups including the array is 13 and is smaller than the preset number 20, and a test environment is created for the array including the array. Specifically, the test environment requirement information of the first test case set in the array divided in table 1 is set 3 (corresponding to cpu is 4, mem is 2, disk is 50, 10), set 4 (corresponding to cpu is 8, mem is 8, disk is 50, 10, 20) and set 6 (corresponding to cpu is 4, mem is 8, disk is 10), and the test environment created is corresponding to cpu is 8, mem is 8, disk is 50, 10, 20. In this way, the test environment created correspondingly can meet the machine resources required by the 3rd, 4 th and 6 th groups in table 1. Thus, only one test environment is needed to be created for one containing array, the accurate number of the created test environments can be further obtained, and machine resources for preparing the test environment resources are saved.

S1063, under the condition that the total number of test cases of each first test case set in the containing array is larger than the preset number, creating a number of test environments corresponding to the ratio for the first test case sets in the containing array according to the ratio of the total number to the preset number.

Illustratively, there are three test cases in Table 1, group 3, testcase-testcase, respectively. The number of test cases in the 6 th group is four, which are testcase to testcase respectively. And six test cases of the 5 th group are testcase, 10-testcase respectively. Thus, the total number of test cases in the three-group first test case set containing the array is 13, which exceeds the preset number 10. The ratio of the total number 13/the preset number 10 is 1.3, and the value of the comparison value is the decimal value carry, so that two test environments need to be created for the inclusion array. At this time, the 5 th group array and the 6 th group array have 10 test cases in total, and the sum of the test cases of the two groups is equal to 10 preset numbers, a test environment is created for the test cases in the first test case sets of the 5 th group and the 6 th group, and the test environment is cpu 8, mem8, disc50, 10 and 20. Simultaneously, a test environment is independently created for the first test case set of the 3 rd group, and the test environment is CPU 4, mem2, disk50 and 10, so that the test environment requirements of test cases in the first test case set of the 3 rd group are met. Thus, one containing array has three first test case sets, but only two test environments are needed to be created to meet the test execution requirements, so that on the premise of meeting the test efficiency, test resources are integrated, and the test environment resources required to be configured are reduced.

In one possible implementation manner, aiming at the situation that no disk information is needed in the test environment, that is, the situation that the number of disks and the size of the disks needed in the test environment of the test cases of the first test case set are 0, for example, the first test case set in the 5 th group in table 1, the test environment does not need to configure a disk when executing, at this time, the disk attribute is null (that is, no disk information), the disk information is recorded as 0, because the first test case set in the 5 th group does not have disk information, the 5 th test environment cannot include a calculation step through a set mainly including disk information, the processing mode aiming at the test cases of the group can be independently processed and stored, and in the subsequent processing process, the needs of the test cases of the group in other hardware resources (such as a CPU and a memory) are distributed to the group where the other test environments are located, so as to save resources and improve the test efficiency.

For the first test case set of which the test cases do not need disk information, the test cases in the first test case set of which the disk information is not needed are distributed to the first test case set under the exclusive array and the containing array, wherein the processor and the memory configuration information needed by the first test case set for receiving the distributed test cases are more than or equal to the maximum value of the processor and the memory configuration information in the needed test environment of the distributed test cases.

For example, with respect to the data in Table 1, we can obtain mutually exclusive arrays after the set inclusion partition, namely, group 1 (corresponding to CPU being 2, mem being 4) and group 2 (corresponding to CPU being 4, mem being 4), and include arrays, namely, group 3 (corresponding to CPU being 4, mem being 2), group 4 (corresponding to CPU being 8, mem being 8) and group 6 (corresponding to CPU being 4, mem being 8). The number of CPU cores required in the test environment of the first test case set of the 5 th group in table 1 is 4, and the memory size is 2, so that the 5 th group of test cases can be accepted as the 2 nd group, the 3 rd group, the 4 th group and the 6 th group, and then the 6 test cases in the 5 th group can be sequentially incorporated into the 2 nd group, the 3 rd group, the 4 th group and the 6 th group.

As can be seen in connection with Table 1, by grouping test cases, we can identify which test cases do not require disk information, such as group 5 arrays. Aiming at the test cases without disk information, analyzing according to the processor and memory configuration information required by each group, finding out the maximum demand value, equally dividing the test cases without disk information into a mutual exclusion array and a containing array, and ensuring that the allocated mutual exclusion array and the processor and memory configuration information containing the array are larger than or equal to the maximum value in the test environment required by the group so as to ensure that the test cases can be executed correctly under the allocated resources. Therefore, through dynamic adjustment and balanced allocation, each test case can be ensured to obtain the computing resources (processors and memories) meeting the requirements of the test case, and resource waste and shortage are avoided. Meanwhile, the arrays reaching the preset number are avoided, resources are configured according to the test cases with the largest requirements, efficient operation of all the test cases is ensured, and the overall test time is shortened.

In this way, through the dual judgment of the disk resource containing relationship and the number of test cases in steps S1061-S1063, the number of the first test case sets and the number of the containing arrays in the obtained mutually exclusive arrays are the number of test environments to be created, so that the accurate number of test environments is obtained, and the diversity of the test cases and the required specific side test environments are fully considered. That is, the number of the first test case sets in the mutex array and the number of the processed array are the machine physical resource data (test environment) to be created, so that the management and optimization of the test environments of the first test case sets are realized, the test environment resources required by the first test case sets are optimized and integrated, the repeated configuration of the same or partially same test environments can be avoided, the redundant test environments are reduced, the accurate test environment number is obtained, the deployment time is shortened, and meanwhile, the test efficiency and the resource utilization rate are both considered. Moreover, the accurate quantity of the test environment requirements is calculated through the algorithm, so that the problems of excessive waste of resources or overlong execution time caused by manual estimation can be avoided.

S107, executing the test case.

In the step, a resource management tool or an automation platform, such as a container technology Kubernetes (container orchestration engine), a Docker Swarm (Docker cluster) and the like, can be used for creating the resources of the test environment, so that quick deployment and resource isolation are realized, and smooth execution of the test case in the required environment is ensured. For example, a test environment may be created in a resource management tool from a plurality of test cases being executed to concurrently execute the plurality of test cases. Thus, time can be saved and the overall test efficiency can be improved.

Or a test framework tool can be deployed to automatically create and schedule the test environment, such as an automation platform JUnit, testNG, pytest, selenium, appium, ansible, puppet, chef, to automatically complete the creation and scheduling of the test environment and execute the test cases.

In some possible implementations, as shown in fig. 5, a test method provided in the embodiment of the present application may further include S108 and S109. Illustratively, S108 and S109 may be performed after S106.

S108, when the test case is executed successfully, releasing the test environment.

In the step, after the test case is successfully executed, the test environment is released, so that the cleanness and stability of the test environment of the operating system can be ensured, and a developer can dynamically schedule and manage the test environment according to different test requirements without influencing the execution of the test case in the next test environment. Meanwhile, the operating system test environment is scheduled according to the requirement, so that serious pollution to the test environment caused by excessive use can be avoided. After each test is finished, the operating system environment which is not needed any more can be released in time, the released test environment can be used for constructing the next test environment, the resources are prevented from being in an idle state for a long time, reasonable application of the resources is ensured, the utilization rate of the resources is improved, and meanwhile, the test efficiency is also improved.

And S109, reserving the test environment when the test case fails to be executed, and releasing the test environment when the reserved time length reaches a preset time length.

In the step, the environment of the successful use case group is cleared, and the environment of the failed use case group is reserved for a period of time as required so as to provide positioning use for the developer. Thus ensuring reasonable application of the environment. For example, the preset time may be 5min, 1 hour or 5 hours, and the developer may set a reasonable retention time according to the historical data and experience to balance the troubleshooting and the resource recovery, so that the sufficient troubleshooting time can be ensured, and the resource is not occupied for a long time.

In this step, preliminary fault localization and analysis may be performed in the retained test environment using automated tools and logs to identify possible causes of failure. The tester can manually enter the reserved environment for detailed problem diagnosis and reproduction operations when necessary.

Specifically, after the tester determines the specific cause of the test case failure (such as finding a code defect or configuration error) and records the related information, the test environment can be released manually or automatically. And (3) overtime release, namely if the failure reason is not determined within the preset time, automatically releasing the test environment by the system to ensure that the resources are not occupied for a long time. Therefore, through dynamic management of the environment, the environment is reserved only when the test fails and diagnosis is needed, and the environment is released in time under other conditions, so that the waste of resources is avoided. Moreover, the environment of the successful test case is released, so that the environment can be rapidly reassigned to a new test task, the environment preparation time is reduced, and the parallel test capability is improved.

According to the testing method provided by the embodiment of the application, the number of the accurate testing environments can be calculated by reasonably grouping the testing environments required by the testing cases and sorting and processing the aggregate containing relation, so that the problem of low execution efficiency caused by resource waste or resource creation too little because the number of the environments cannot be accurately prepared is solved.

And then, constructing different flexible and controllable test environment sets by utilizing the physical machines and the virtual machines according to the number of the test environments, and realizing the capability of preparing the automatic test environments. And then an automatic test framework (automatic test platform) is built, the test environment can be dynamically scheduled and managed according to the test requirement, and the test environment is immediately released when the test case is successfully executed, so that the recovery of the test environment is realized, and a powerful base capacity is provided for the test of the linux operating system. Thus, the method realizes balance between cost and resource utilization, and avoids the problems of excessive waste or excessive cost. Finally, the embodiment not only can meet the test service of the operating system, but also adds the limit of the execution quantity of the test cases under each test environment under the condition of sufficient environment resources so as to better control the test execution efficiency.

It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not mean the order of execution, and the execution order of the processes should be determined by the functions and the internal logic, and should not limit the implementation process of the embodiments of the present application. In addition, in some possible implementations, each step in the foregoing embodiments may be selectively performed according to practical situations, and may be partially performed or may be performed entirely, which is not limited herein. In addition, all or part of any features of any of the above embodiments may be freely and arbitrarily combined without contradiction, and the combined technical solutions are also within the scope of the present application.

Example 1

The embodiment 1 of the application provides a test method, which aims at dispatching and executing a use case test aiming at an execution environment of a Linux kernel operating system. As shown in fig. 6, the test method includes the steps of:

S51, receiving a test task.

Here, reference may be made to step S101 in fig. 2, and details are not repeated here.

S52, acquiring test case information.

In the step, test case information is obtained through a test task, wherein when the test case is programmed, a developer records test environment information required by each test case, including the number of CPU cores, the memory size, the disk space, the network card setting and the like.

S53, arranging and combining the test cases.

Referring to steps S103-S104 in fig. 2, the number of the accurate test environments is obtained according to the number of the first test case sets and the number of the inclusion arrays in the mutex array, which are obtained by performing test case arrangement and combination and performing set inclusion calculation.

S54, creating test environments according to groups.

In the step, according to the number of the acquired accurate test environments, a test environment is created according to a physical machine or a virtual machine.

S55, deploying a test framework.

In this step, the developer may deploy the test framework tool to perform an automated test environment deployment and scheduling, for example, an automated platform such as JUnit, testNG, pytest, selenium, appium, ansible, puppet, chef, to automatically complete the scheduling of the test environment.

S56, executing the test cases concurrently.

In this step, the test cases are executed by the test framework deployed in step S55.

S57, judging whether the test case is successfully executed.

S58, releasing the test environment.

And when the test case is successfully executed, immediately releasing the test environment.

S59, reserving a test environment.

Under the condition that the test case fails to be executed, the test environment is reserved for a period of time, and the test environment reservation time can be reasonably set according to historical data and experience so as to balance troubleshooting and resource recycling.

S60, positioning analysis.

And in the retention time of the test environment, positioning and analyzing the reasons of the failure of the execution of the test cases so as to remove faults, so that the test cases can be successfully executed, and the test of the operating system is completed.

S61, manually releasing.

After determining the cause of the test case execution failure, the test environment may be manually released.

S62, releasing due.

And the test environment can be automatically released after the preset time is reserved without manual operation after the reason of the failure of the execution of the test case is determined, so that the reasonable application of the environment is ensured.

According to the testing method provided by the embodiment 1 of the application, the accurate environment quantity is calculated through arranging and combining the environment information (CPU quantity, mem, disk, netcard and the like), the operating system testing environment is established according to the calculated environment information deployment, the accurate provision of the testing environment quantity under the condition of testing environment diversity is realized, the recovery resources can be flexibly established in real time through the operation environment scheduling scheme of the automatic platform, excessive waste of the resources is avoided, and meanwhile, the testing execution efficiency can be controlled successfully at the maximum through the limitation of the number of the testing environment cases.

As shown in fig. 7, the embodiment of the application further provides a testing device, which comprises an acquisition module 1021, a grouping module 1022, a construction module 1023, a testing module 1024 and a monitoring module 1025.

The acquisition module 1021 is used for acquiring a test task, wherein the test task comprises a plurality of test cases. The obtaining module 1021 is further configured to obtain a test environment requirement of each of the test cases. The test case comprises a test environment when the test case is executed on an operating system, wherein the test environment comprises the number of processor cores, the memory size, disk information and network card configuration information required by the test case when the test case is executed.

The grouping module 1022 determines a plurality of first test case sets according to the test environment requirement information of the test cases, wherein the test environment requirements of all the test cases contained in one first test case set are the same. In other words, the grouping module 1022 may group test cases with the same test environment into a group based on the test environment by arranging and combining the test cases. Specifically, test cases with the same test environment requirements are divided into a group according to the number of processor cores, the memory size, the disk information and the network card configuration information required by the test cases during execution.

In one possible implementation manner, the grouping module 1022 is further configured to determine a mutex array and at least one inclusion array according to the plurality of first test case sets, where the mutex array includes at least one first test case set, and no inclusion relationship exists between different first test case sets in the mutex array, and the inclusion array includes the plurality of first test case sets, and no inclusion relationship exists between different first test case sets in the inclusion array.

The construction module 1023 creates a corresponding test environment for each first test case set in the mutex array, and distributes each first test case set to the corresponding test environment, wherein each test environment meets the test environment requirements of the distributed first test case set.

In one possible implementation manner, the construction module 1023 is further configured to create a corresponding test environment for each containing array, and allocate all the first test case sets in each containing array to the corresponding test environment, where each test environment meets the test environment requirements of the allocated first test case set.

The test module 1024 is used to schedule the test environment for executing test cases.

Optionally, the test module 1025 may automatically schedule a test environment required by the multiple test cases according to the multiple test cases, and concurrently execute the multiple test cases.

The monitoring module 1025 is used for monitoring the execution result of the test case in real time, releasing the test environment when the test case is successfully executed, reserving the test environment when the test case is failed to be executed, and releasing the test environment when the reserved time length reaches the preset time length.

As one implementation, the monitoring module 1026 is further configured to determine a cause of the test case execution failure when the test case execution fails.

Thus, the computing device provided by the embodiment of the application has the capability of automatic environment preparation, and can dynamically schedule and manage the test environment according to the test requirement. In addition, the embodiment needs to consider the balance of cost and resource utilization, can accurately acquire the number of test environments required by the test through calculation, and can timely release the test environments after the test cases are executed, so that the problems of excessive waste or excessive cost are avoided.

As shown in fig. 8, a computing device is also provided in an embodiment of the present application, and the computing device 600 may be the management device 101 in fig. 1. By way of example, the computing device 600 may be a server or a terminal.

The computing device 600 includes a bus 601, a processor 602, a memory 603, and a communication interface 604. The processor 602, the memory 603 and the communication interface 604 communicate via a bus 601. Computing device 600 may be a server or a terminal device. It should be understood that the present application is not limited to the number of processors, memories in computing device 600.

Bus 601 may be a peripheral component interconnect standard (PERIPHERAL COMPONENT INTERCONNECT, PCI) bus or an extended industry standard architecture (extended industry standard architecture, EISA) bus, or the like. The buses may be divided into address buses, data buses, control buses, etc. For ease of illustration, the figures are shown with only one line, but not with only one bus or one type of bus. The bus 601 may include a path to transfer information between various components of the computing device 600 (e.g., the processor 602, the memory 603, the communication interface 604).

The processor 602 may include any one or more of a central processing unit (central processing unit, CPU), a graphics processor (graphics processing unit, GPU), a Microprocessor (MP), or a digital signal processor (DIGITAL SIGNAL processor, DSP).

The memory 603 may include a nonvolatile memory (non-volatile memory), such as a read-only memory (ROM), a flash memory, a mechanical hard disk (HARD DISK DRIVE, HDD), or a Solid State Disk (SSD) STATE DRIVE.

The memory 603 has stored therein executable program instructions that are executed by the processor 102 to implement the test methods respectively referred to in the above embodiments.

The communication interface 604 enables communication between the computing device 600 and other devices or communication networks using a transceiver module such as, but not limited to, a network interface card, transceiver, or the like.

The embodiment of the application also provides a computer readable storage medium. The computer readable storage medium is for storing computer program instructions which, when run on a computing device, cause the computing device to perform the test methods referred to in the above embodiments. The computer readable storage medium may be a magnetic medium, (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk), etc.

Embodiments of the present application also provide a computer program product comprising instructions. The computer program product, when run on a computing device, causes the computing device to perform the test methods referred to in the embodiments above.

Finally, the above embodiments are only used to illustrate the technical solution of the present application. It will be appreciated by those skilled in the art that, although the application has been described in detail with reference to the foregoing embodiments, various modifications may be made to the technical solutions described in the foregoing embodiments, or equivalents may be substituted for some of the technical features thereof. Such modifications and substitutions do not depart from the spirit and scope of the corresponding technical solutions in the various embodiments of the application.

Claims

1. A method of testing, comprising:

Acquiring a test task, wherein the test task comprises a plurality of test cases;

acquiring the test environment requirement of each test case;

determining a plurality of first test case sets according to the test environment demand information, wherein the test environment demands of all test cases contained in one first test case set are the same;

Determining a mutual exclusion array and at least one inclusion array according to the plurality of first test case sets, wherein the mutual exclusion array comprises at least one first test case set, and no inclusion relationship exists among different first test case sets;

creating a corresponding test environment for each first test case set in the mutex array, and distributing each first test case set to the corresponding test environment, wherein each test environment meets the test environment requirements of the distributed first test case set;

Creating a corresponding test environment for each containing array, and distributing all first test case sets in each containing array to the corresponding test environment, wherein each test environment meets the test environment requirements of the distributed first test case sets;

And executing the test task.

2. The method of claim 1, wherein the test environment requirements include processor, memory, and disk information.

3. The method of claim 2, wherein the determining the mutually exclusive array and the at least one containing array from the plurality of the first test case sets comprises:

And determining a mutual exclusion array and at least one inclusion array based on disk information in the test environment required by the first test case.

4. The method of claim 2, wherein the disk information includes a number of disks and a disk size, wherein the determining the mutex array and the at least one include array based on disk information in the test environment required by the first test case comprises:

traversing each first test case set;

Under the condition that a first test case set has a containing relation with the number of disks and the size of the disks required by any one first test case set, the first test case set and the first test case set with the containing relation are taken as a containing array;

And under the condition that the first test case set has no inclusion relation with the number of disks and the size of the disks required by any one first test case set, putting the first test case set into a mutex array.

5. The method of any of claims 1-4, wherein the creating a corresponding test environment for each first set of test cases in the mutex array comprises:

Creating the processor core number of the test environment according to the processor core number in the test environment requirement of the first test case set, creating the memory size of the test environment according to the memory size in the test environment requirement of the first test case set, and creating the disk information of the test environment according to the disk information in the test environment requirement of the first test case set.

6. The method of any of claims 1-4, wherein creating a corresponding test environment for each of the containing arrays comprises:

Creating disk information corresponding to the test environment according to the disk information which is most needed by the first test case set in the containing array, creating the processor core number corresponding to the test environment according to the processor core number which is most needed by the first test case in the containing array, and creating the memory size corresponding to the test environment according to the memory size which is most needed by the first test case in the containing array.

7. The method of claim 6, wherein creating a corresponding test environment for each of the containing arrays comprises:

determining whether the total number of test cases of each of a plurality of first test case sets in the array is smaller than or equal to a preset number;

Under the condition that the total number of the test cases of the plurality of first test case sets in each containing array is smaller than or equal to the preset number, creating a test environment for each containing array, and distributing all the first test case sets in each containing array to the corresponding test environments;

Under the condition that the total number of the test cases of each first test case set containing the array is larger than the preset number, a test environment with the number corresponding to the ratio is created for the first test case set containing the array according to the ratio of the total number to the preset number.

8. The method according to any one of claims 1-7, further comprising:

Releasing the test environment when the test case is successfully executed;

And when the execution of the test case fails, reserving the test environment, and when the reserved time length reaches a preset time length, releasing the test environment.

9. The method of any of claims 1-8, wherein the test environment requirements include one or more of network card, GPU, or SATA configuration information.

10. A computing device includes a memory and a processor;

The memory is coupled to the processor;

the memory is used for storing program instructions;

the processor being configured to invoke the program instructions to cause the computing device to implement the method of any of claims 1-9.