JP7182044B2 - Test management server and system - Google Patents

Description

The present invention relates to test management servers and systems.

In the development of web services and applications, it is common to repeatedly test how the finished system actually works and whether it behaves as expected. The number of possible operations that a user can perform on a system is enormous, and it is neither efficient nor practical to manually test each one. Therefore, test automation is in progress.

For example, Patent Literature 1 discloses an automatic test tool that performs a screen display operation test associated with an operation. In Patent Literature 1, images of test results are displayed for comparison.

JP 2012-038162 A JP-A-2003-058370

https://software.microfocus.com/ja-jp/solutions/software-development-lifecycle, retrieved on July 9, 2018 https://software.microfocus.com/ja-jp/products/unified-functional-automated-testing/overview, retrieved on July 9, 2018

However, in the technique disclosed in Patent Literature 1, when registering a test case, it is necessary for the user to actually perform an operation related to the test, such as pressing a button. Such registration of test cases is troublesome for the user and cannot be said to be highly convenient for the user.

SUMMARY OF THE INVENTION The present invention has been made in view of these problems, and an object thereof is to provide a technology that can reduce the user's labor when setting test procedures.

One aspect of the invention relates to a test management server. This test management server includes test procedure reception means for accepting a test procedure prescribed using an item logical name for allowing a person to recognize items included in a screen to be tested, and a computer for receiving an item corresponding to the item logical name. repository reception means for receiving the repository information of the screen to be tested, including the physical name of the item to be recognized by the tester; and generating a script for executing the test based on the received test procedure and the received repository information. means for determining whether either the corresponding item physical name or the item logical name in the repository information of the screen to be tested has been changed; and the corresponding item physical name and item logical name is determined to have changed, the unchanged item of the corresponding item physical name and item logical name and the changed item of the corresponding item physical name and item logical name and display means for displaying the items before and after the change .

It should be noted that any combination of the above constituent elements, or mutual replacement of the constituent elements and expressions of the present invention with devices, methods, systems, computer programs, recording media storing computer programs, etc., are also included in the present invention. It is effective as an aspect of

According to the present invention, it is possible to reduce the user's trouble in setting the test procedure.

1 is a schematic diagram showing an overview of a test platform according to an embodiment; FIG. 1 is a schematic diagram showing the configuration of a test platform according to an embodiment; FIG. FIG. 3 is a schematic diagram for explaining automatic generation processing and management of a test script in the test management server of FIG. 2; FIG. 11 is a representative screen diagram of a part of a test case (summary) input screen; FIG. 10 is a representative screen diagram of a part of the test case (detailed version) input screen; FIG. 4 is a schematic diagram for explaining repository information; 3 is a hardware configuration diagram of a test management server in FIG. 2; FIG. 3 is a block diagram showing the functions and configuration of the test management server of FIG. 2; FIG. 3 is a representative screen view of a test management screen displayed on the display of the test management server of FIG. 2; FIG. 3 is a representative screen diagram of a script display screen displayed on the display of the test management server of FIG. 2; FIG. 3 is a representative screen diagram of a test execution screen displayed on the display of the test management server of FIG. 2; FIG. 3 is a representative screen diagram of an execution details designation screen displayed on the display of the test management server of FIG. 2; FIG. 3 is a representative screen view of a test result screen displayed on the display of the test management server of FIG. 2; FIG. 3 is a representative screen diagram of a verification result notification screen displayed on the display of the test management server of FIG. 2; FIG. FIG. 3 is a flow chart showing a series of processes in the test management server of FIG. 2; FIG. FIG. 11 is a schematic diagram showing the configuration of a test platform according to a first modified example; FIG. 11 is a schematic diagram showing the configuration of a test platform according to a second modified example;

Hereinafter, the same or equivalent constituent elements, members, and processes shown in each drawing are denoted by the same reference numerals, and duplication of description will be omitted as appropriate. Also, in each drawing, some members that are not important for explanation are omitted.

FIG. 1 is a schematic diagram showing an outline of a test platform 2 according to an embodiment. The test platform 2 supports the efficiency of the test process, from test automation preparation to test implementation and management. The test platform 2 uses an automatic generation tool 4 to prepare for test automation. Specifically, the user creates test specifications using the automatic generation tool 4 . The created specifications are automatically converted into test scripts for test execution by the automatic generation tool 4 .

Management tools 6 manage test execution and test assets. The management tool 6 may be implemented using, for example, ALM (Application Lifecycle Management) (see Non-Patent Document 1). Management of test execution can be done anytime and anywhere, and the progress of test execution is also managed. Managed test assets include test specifications, test scripts, and trails.

The automatic test tool 8 executes the test (keying/verification). The automatic test tool 8 may be implemented using, for example, UFT (Unified Functional Testing) (see Non-Patent Document 2). The automatic test tool 8 automatically performs keystrokes based on the test script and collects the traces. The automatic test tool 8 makes verification efficient by the method according to the present embodiment.

FIG. 2 is a schematic diagram showing the configuration of the test platform 2 according to the embodiment. The test platform 2 comprises a test management server 10 , a client 12 , a test target 14 and a file server 16 . Test management server 10 functions as an interface with testers 18 . Each component is communicably connected to each other via a network (not shown) such as the Internet. The test management server 10 prepares a test script 20 and input data 22 for testing resources on the network (such as data and screens specified by a URL (Uniform Resource Locator)) based on input from a tester 18. Generate and keep. In this embodiment, the test management server 10 has a testware management function and a script generation function. However, the present invention is not limited to this. I will explain the case where it is implemented in

A series of operations in the test platform 2 are as follows.
1. A tester 18 instructs the test management server 10 to execute a test.
2. The test management server 10 transmits the test script 20 under management to the client 12 for remote execution on the client 12 .
3. Execution engine 24 of client 12 operates browser 26 to execute the test. The execution engine 24 returns execution results to the test management server 10 .
4. The file server 16 collects traces of the client 12 and the test target 14 generated by the execution of the test script 20 (for example, captured images of screens displayed on the browser 26, operation logs of the test target 14, etc.).
5. The comparison engine 28 of the file server 16 performs the comparison, comparing the prearranged expectations with the collected evidence.
6. File server 16 registers comparison results and trails with test management server 10 .
7. Test management server 10 communicates test results 30 to tester 18, including execution results, trails, and comparison results.

FIG. 3 is a schematic diagram for explaining automatic test script generation processing and its management in the test management server 10 of FIG. For example, in a web service development project 32, web service design information 34 is generated. Also, a server-side system 36 is constructed for realizing the web service. The tester 18 refers to the design information 34 and creates a test case (summary) 38 describing the outline of the test.

FIG. 4 is a representative screen view of part of the input screen 400 of the test case (summary) 38. As shown in FIG. The tester 18 accesses the test management server 10 from his own terminal, and the test management server 10 returns the input screen 400 to the terminal of the tester 18 . The person in charge of testing 18 inputs the outline of the test to be performed on the input screen 400 . In particular, the person in charge of testing 18 inputs what kind of test is to be performed on the screen to be tested. The test management server 10 acquires the data input to the input screen 400 and inputs it to the management tool 6 .

Returning to FIG. 3, the tester 18 generates a test case (detailed version) 40 that describes in detail data to be input based on the test case (summary) 38 and test procedures. The test cases (detailed version) 40 may be generated using other human resources (offshore resources, etc.) instead of the tester 18 .

FIG. 5 is a representative screen diagram of a part of the input screen 402 of the test case (detailed version) 40 . The tester 18 accesses the test management server 10 from his own terminal, and the test management server 10 returns an input screen 402 to the terminal of the tester 18 . The tester 18 inputs test procedures and data described in the test case (summary) 38 on the input screen 402 .

The input screen 402 includes a serial number that defines the order in which the test is performed, the test category, the test target, the logical name of the screen to be tested, the physical name of the screen to be tested, and the screen to be tested. It is configured such that an item logical name for allowing a person to recognize the included item, a verification category, and a value for verification are input in association with each other. Operation or verification can be selected as the test category. The action may be, for example, clicking a UI element on the screen, entering text into a text box, or the like. Verification may be, for example, determining whether a value returned from system 36 as a result of an operation matches or does not match an expected value.

The verification category specifies the type of verification when the test category is verification. For example, if it is set to "=", it will be Pass if there is a complete match with the value for verification, otherwise it will be Fail. Alternatively, if it is set to "Contain", it will pass if it contains a value for verification, otherwise it will fail. The test management server 10 acquires the data entered on the input screen 402 .

Returning to FIG. 3, the test management server 10 acquires the screen information (repository information) of the test object from the system 36 of the project 32 . The test management server 10 accesses the screen to be tested and acquires the repository information 42 by analyzing the code of the screen. The test management server 10 automatically generates a test script for test execution based on the test case (detailed version) 40 and the repository information 42 and inputs it to the management tool 6 .

The tester 18 separately sets the test environment for the management tool 6 . In the management tool 6, test environment settings, test cases (summaries), test scripts, and test data (also called input data) are associated with each other, but managed separately.

In the execution/verification phase, test scripts and test data are distributed to a test environment, tests are executed against various combinations of OS (Operating System) and browsers, and the results are collected and verified.

FIG. 6 is a schematic diagram for explaining repository information. In FIG. 6, a language 46, which is an item on the book information search input screen 44, is given an item logical name of "language". A repository display screen 48 displays the repository information of the book information search input screen 44 . In this repository information, the item physical name "BOOKLANGUAGE" is given to the language 46, which is the item of the book information search input screen 44, and the item logical name "Language" and the item physical name "BOOKLANGUAGE" correspond to each other. indicates that The item physical name is an item name that allows the computer to recognize the item. For example, auto-generated test scripts specify items by item physical name. On the other hand, the item logical name may be a character string written in a predetermined language such as Japanese, English, or Chinese.

FIG. 7 is a hardware configuration diagram of the test management server 10 of FIG. The client 12, test target 14, and file server 16 may have a hardware configuration similar to that shown in FIG. Test management server 10 includes memory 130 , processor 132 , communication interface 134 , display 136 and input interface 138 . These elements are each connected to bus 140 and communicate with each other via bus 140 .

The memory 130 is a storage area for storing data and programs. Data and programs may be stored in memory 130 permanently or temporarily. In particular, the memory 130 has an area allocated as a test information holding section 114, which will be described later. The processor 132 realizes various functions in the test management server 10 by executing programs stored in the memory 130 . The communication interface 134 is an interface for transmitting and receiving data to and from the outside of the test management server 10 . For example, communication interface 134 includes an interface for accessing network 6 . The display 136 is a device for displaying various information, such as a liquid crystal display or an organic EL (Electroluminescence) display. The input interface 138 is a device for receiving input from the user. The input interface 138 includes, for example, a mouse, a keyboard, and a touch panel provided on the display 138 .

FIG. 8 is a block diagram showing the functions and configuration of the test management server 10 of FIG. Each block shown here can be implemented by hardware such as a computer CPU and other elements and mechanical devices, and is implemented by software such as a computer program. It depicts the function blocks to be used. Therefore, those skilled in the art who have read this specification will understand that these functional blocks can be implemented in various ways by combining hardware and software.

The test management server 10 includes a test procedure reception unit 102, a repository reception unit 104, a script generation unit 106, a test management unit 108, a test execution unit 110, a test result provision unit 112, and a test information storage unit 114. , provided. The test information holding unit 114 functions as a master database and holds the test script 20, input data 22, and test results 30 shown in FIG. Also, the test information holding unit 114 holds the setting of the test environment, the test case (summary), the test script, and the test data shown in FIG. 3 in association with each other.

The test procedure receiving unit 102 receives a test procedure defined using the item logical names of the items included in the screen to be tested. As described above with reference to FIG. 5, the test procedure reception unit 102 acquires the test procedure input by the tester 18 to the input screen 402 of the test case (detailed version) 40 .

A repository reception unit 104 receives repository information of a screen to be tested. As described above with reference to FIG. 3, the repository reception unit 104 obtains the repository information of the screen by obtaining and analyzing the screen to be tested. As described above with reference to FIG. 6, the repository information includes correspondences between item logical names and item physical names of items included in the screen.

The script generation unit 106 generates a test script and input data for executing the test based on the test procedure received by the test procedure reception unit 102 and the repository information received by the repository reception unit 104 . The script generation unit 106 converts the item logical name used in the test procedure into the corresponding item physical name by referring to the repository information. The script generation unit 106 generates a test script based on the test procedure after the conversion, using a known script automatic generation technology such as that described in Patent Document 2, for example. The script generation unit 106 registers the generated test script and input data in the test information holding unit 114 .

For example, referring to the example shown in FIG. 6, the value "3" (corresponding to "Chinese version") of the item logical name "language" is specified in the test procedure received by the test procedure receiving unit 102. , is included. The script generation unit 106 refers to the repository information of the book information search input screen 44 and converts the item logical name "Language" of the command into the corresponding item physical name "BOOKLANGUAGE". As a result of the conversion, the command specifies the item value "3" of the item physical name "BOOKLANGUAGE".

In this embodiment, a case will be described in which the values of items in the test procedure are specified by physical values (values for recognition by a computer). In another embodiment, the value of an item may be designated as a logical value (a value for human recognition), and the script generation unit 106 may refer to repository information to convert the logical value into a physical value. For example, referring to the example shown in FIG. 6, the test procedure accepted by the test procedure accepting unit 102 includes a command to specify the logical value “Chinese version” of the item with the item logical name “language”. Suppose you are The script generation unit 106 refers to the repository information on the book information search input screen 44, and replaces the item logical name "Language" of the command with the corresponding item physical name "BOOKLANGUAGE" and the logical value "Chinese version" as the physical value "3". , respectively. As a result of the conversion, the command specifies the physical value "3" of the item with the item physical name "BOOKLANGUAGE".

A test management unit 108 functions as a management tool 6 . FIG. 9 is a representative screen view of the test management screen 404 displayed on the display 136 of the test management server 10. As shown in FIG. The test management unit 108 causes the display 136 to display the test management screen 404 upon request from a user such as the person in charge of testing 18 . After the script generation unit 106 creates a test script and registers it in the test information holding unit 114, the created test script and procedure are managed by test numbers on the test management screen 404. FIG. The test management screen 404 includes a case selection area 406 for selecting a test case to be displayed, a plurality of tabs 408 for selecting display contents of the selected test case, and information corresponding to the selected tab. and a display area 410 for displaying . In the example of FIG. 9, the test case with the test number "R1-R1-1-1-1-1-1" is selected in the case selection area 406, and the test cases (summary) corresponding to the multiple tabs 408 are displayed. The tab “Design Steps” is selected, and the information of the test case (summary) 38 created and registered by the tester 18 is displayed in the display area 410 .

FIG. 10 is a representative screen view of the script display screen 414 displayed on the display 136 of the test management server 10. As shown in FIG. When the user selects the tab "Test Script" included in the plurality of tabs 408 on the test management screen 404 of FIG. A script display screen 414 has a case selection area 406 , a plurality of tabs 408 , and a script display area 412 for displaying test scripts generated and registered by the script generation unit 106 . In this way, the test script is also managed with the same test number as the test case (summary).

The test execution unit 110 functions as an automatic test tool 8. FIG. FIG. 11 is a representative screen view of the test execution screen 416 displayed on the display 136 of the test management server 10. As shown in FIG. The test execution unit 110 displays the test execution screen 416 on the display 136 upon request from the user. The test execution screen 416 is for selecting a test case to be executed from the test cases held in the test information holding unit 114 and executing the test. The test execution screen 416 displays a folder selection area 418 for selecting a folder in which test cases to be executed are stored, and information on the test cases stored in the folder selected in the folder selection area 418, and displays and executes the test cases. It has a test display area 420 , an individual execution button 422 , and an overall execution button 424 that enable designation of a target test case.

For example, when the user designates one test case displayed in the test display area 420 with the cursor 426 and clicks the individual execution button 422, the test execution unit 110 transmits the test script of the designated test case to the client 12. command to run the test. When the user clicks the run all button 424 , the test execution unit 110 causes the client 12 to run the series of test cases displayed in the test display area 420 .

FIG. 12 is a representative screen diagram of the execution details designation screen 428 displayed on the display 136 of the test management server 10. As shown in FIG. When the user clicks the individual execution button 422 or all execution button 424 on the test execution screen 416 shown in FIG. The execution details designation screen 428 displays the test name 430 of the test selected as the execution target, the host name 432 of the host that executes the test, and the test status 434 in association with each other.

The user can specify an arbitrary host by clicking the host name 432 cell on the execution details specification screen 428 . For example, when the user clicks on cell 436 of host name 432 in FIG. 12, test execution unit 110 causes display 136 to display a list of hosts (or clients) that can execute the corresponding test script. When the user selects a desired host from the displayed list, test execution unit 110 displays the host name of the selected host in cell 436 . When the test execution button 438 on the execution details designation screen 428 is clicked, the test execution unit 110 causes the selected host to execute the test script of the test designated by the test name 430 . In this way, the host on which the test is to be executed can be specified on a case-by-case basis, so the test can be executed simultaneously by multiple hosts, and the test can be completed more quickly.

The test result providing unit 112 provides the user with the test results 30 obtained from the client 12 or the file server 16 that executed the test script. FIG. 13 is a representative screen view of the test result screen 440 displayed on the display 136 of the test management server 10. As shown in FIG. The test result providing unit 112 causes the display 136 to display the test result screen 440 upon request from the user. The test result screen 440 includes the presence/absence of a captured image of the screen displayed on the browser 26, an execution ID for specifying test execution, an execution name, a test name of the executed test, a test state, and a test result. The time taken, the date the test was executed, the time the test was executed, and the host name of the host that executed the test are displayed in association with each other. When the user clicks a cell in which the presence or absence of a captured image is "present", an evidence folder (not shown) is displayed. Evidence folders are provided for each test case and store evidence such as captured images and logs.

FIG. 14 is a representative screen diagram of the verification result notification screen 442 displayed on the display 136 of the test management server 10. As shown in FIG. As described with reference to the input screen 402 in FIG. 5, the test case (detailed version) 40 may include verification of operation results in addition to operations on the screen. A script generation unit 106 generates a test script including verification. Therefore, when the test script is executed, the verification will also be performed automatically. The test result providing unit 112 extracts the verification result from the execution result obtained from the client 12 that executed the test script. The test result providing unit 112 causes the display 136 to display a verification result notification screen 442 including the acquired verification result. The verification result notification screen 442 displays verification contents (actions), evidence, checkpoints, verification results, and verification details in association with each other. The verification result is "PASS" if the output matches or corresponds to the expected value, and "FAIL" if not.

The operation of the test management server 10 configured as above will be described.
FIG. 15 is a flow chart showing a series of processes in the test management server 10 of FIG. The test management server 10 receives an input of a test procedure from the user (S502). The test management server 10 accesses the screen to be tested and acquires the screen code (eg, HTML (HyperText Markup Language) code) (S504). The test management server 10 acquires the repository information of the screen to be tested by analyzing the acquired code (S506). The test management server 10 refers to the repository information acquired in step S506, and converts the item logical name in the test procedure received in step S502 into an item physical name (S508). The test management server 10 generates a test script based on the test procedure after conversion in step S508 (S510). The test management server 10 stores the generated test script in the test information holding unit 114 (S512).

In the above-described embodiments, examples of holding units are hard disks and semiconductor memories. In addition, based on the description of this specification, each part is implemented by a CPU (not shown), an installed application program module, a system program module, a semiconductor memory that temporarily stores the contents of data read from the hard disk, etc. Those skilled in the art who have access to this specification will understand what is possible.

According to the test platform 2 according to the present embodiment, a test script is automatically generated based on a test procedure and repository information defined using item logical names. In this generation, the item logical name is converted to the item physical name by referring to the repository information. Therefore, when the tester 18 registers the test case (detailed version) 40 from the test case (summary) 38, there is no need to know or input the item physical name. Alternatively, the tester 18 can register the procedure of the test with a more descriptive item logical name. Therefore, the tester 18 can register the test case (detailed version) 40 more quickly and accurately, thereby improving the efficiency of the test.

Normally, the item physical name is decided only when the actual screen is completed, so with the conventional method of specifying the test procedure by the item physical name, it was not possible to create the test procedure until the actual screen was completed. rice field. On the other hand, in the test platform 2 according to the present embodiment, it is not necessary to specify the item physical name in the test case (detailed version) 40, so the test procedure can be created without waiting for the completion of the actual screen. be able to. For example, even if the screen is under development, the test case (detailed version) 40 can be created as soon as the design information 34 is decided. This allows for shorter test cycles and faster service releases.

Depending on the test target, the repository information may be modified, added, or deleted due to changes in the development specifications of the test target, correction of bugs, enhancement after operation, etc. Either or both of the item physical name and the item logical name may be modified. The repository reception unit 104 acquires the repository information again at an appropriate timing (timing to re-execute the test, timing to start the test management server 10, timing to re-accept the repository from the user), Corrections, additions, and deletions of the repository information may be specified by comparing with the repository information acquired this time. When test execution fails due to modification, addition, or deletion of repository information, it becomes possible to verify that test execution has failed due to modification, addition, or deletion of the identified repository information. Here, if either the item physical name or the item logical name is modified for a certain piece of repository information, the unmodified one is used as a key to identify the modified item before and after modification. For example, if there is repository information where the item physical name has been modified and the item logical name has not been modified, the item logical name is used as a key, and the item physical name before modification and the item physical name after modification are These associations can be used to update a previously created test script and test run, in some cases allowing a failed test run to succeed. Instead of updating this test script automatically, it may be presented to the user in advance and updated upon receiving an instruction, or simply the item physical name before correction and the item physical name after correction may be It may be only an action that is displayed to the user along with the physical name.

The configuration and operation of the test platform 2 according to the embodiment have been described above. Those skilled in the art will understand that this embodiment is merely an example, and that various modifications are possible in combination of each component and each process, and that such modifications are within the scope of the present invention.

Although the test management server 10 has a testware management function and a script generation function in the embodiment, the present invention is not limited to this.

FIG. 16 is a schematic diagram showing the configuration of a test platform 600 according to the first modified example. The test platform 600 implements the test management function in the test management server, and the test execution function and the automatic script generation function are implemented in the client terminal and the verification execution terminal (both may be the same terminal or different terminals). good). Alternatively, the automatic script generation function may be implemented in another terminal in the implementation environment that is different from the client terminal and verification execution terminal. The dashed-dotted arrows in FIG. 16 represent operations performed manually, and the dashed arrows represent operations automatically performed by a tool.

FIG. 17 is a schematic diagram showing the configuration of a test platform 700 according to the second modified example. The test platform 700 omits the test management function and uses the execution environment alone. The test platform 700 implements a test execution function and a script automatic generation function on a client terminal and a verification execution terminal (both may be the same terminal or may be different terminals). Alternatively, the automatic script generation function may be implemented in another terminal in the implementation environment that is different from the client terminal and verification execution terminal. The dashed-dotted arrows in FIG. 17 represent operations performed manually, and the dashed-line arrows represent operations automatically performed by a tool.

2 test platform, 4 automatic generation tool, 6 management tool, 8 automatic test tool.

Claims (5)

a test procedure reception means for accepting a test procedure defined using an item logical name for allowing a person to recognize an item included in a screen to be tested;
repository reception means for receiving repository information of a screen to be tested, including an item physical name for allowing a computer to recognize an item corresponding to the item logical name;
generation means for generating a script for executing the test based on the accepted test procedure and the accepted repository information;
determination means for determining whether either the corresponding item physical name or the item logical name in the repository information of the screen to be tested has been changed;
When it is determined that one of the corresponding item physical name and the item logical name has been changed, the unchanged item of the corresponding item physical name and the item logical name and the corresponding a display means for displaying a changed item of the item physical name and the item logical name before and after the change . 2. The test management server according to claim 1, wherein said generating means converts an item logical name used in a test procedure into a corresponding item physical name by referring to repository information. The test procedure receiving means acquires a test procedure input by a user,
3. The test management server according to claim 1, wherein said repository reception means acquires repository information of a screen to be tested by acquiring and analyzing the screen. The testing procedure includes verification of the operation results,
4. The test management server according to any one of claims 1 to 3, wherein said test management server further comprises providing means for providing a verification result to a user. The test management server further uses the unchanged item of the corresponding item physical name and item logical name as a key to change the changed item of the corresponding item physical name and item logical name. 5. The test management server according to any one of claims 1 to 4, further comprising specifying means for specifying and after the change.

Images