CN1786925A - TTCN-3 testing system basedon C++ mapping and its testing method - Google Patents

Abstract

The invention proposes a TTCN-3 test system and test method based on C++ mapping. The test system and its test method are based on the C++ mapping standard of TTCN-3, and the TTCN-3 test cases are converted into semantically equivalent C++ test cases through the language converter, and compiled, loaded and executed in the TTCN-3 operating environment to achieve Implement testing of the system under test. In the test cases generated by conversion, each test component is realized as a process, which avoids the limitation of the stack space and the number of test components in the commonly used thread technology. The TTCN-3 operating environment realized by the invention can not only satisfy the black-box test of general systems, but also be suitable for the tests of those systems with the characteristics of concurrency, interaction, real-time and the like.

Description

TTCN-3 Test System and Test Method Based on C++ Mapping

technical field

The invention relates to a TTCN-3 test system realized based on C++ mapping, and also relates to a test method utilizing the test system, which belongs to the technical field of software test.

Background technique

TTCN (Testing and Test Control Notation, test and test control notation) is a widely used international standard test system, the latest version is TTCN-3. TTCN-3 can be used to describe various responsive system tests on various communication ports. Typical areas of its application are protocol testing (including mobile and Internet protocols), service testing, platform-based CORBA testing, API testing, and more. TTCN-3 is not limited to conformance testing, it can also be used in many other kinds of testing, such as interoperability testing, robustness testing, regression testing, system and integration testing, etc.

Currently, the existing TTCN-3 operating environments are all implemented based on Java or C mapping methods. Since the Java language needs to have good platform independence, it brings about the problem of relatively low execution efficiency, and it is difficult to meet the test needs of real-time systems in response time. As for the C mapping method, although its execution efficiency is relatively high, the coupling of the TTCN-3 operating environment realized by this method is too tight, which is not conducive to the secondary development of the specific system under test.

In addition, the existing TTCN-3 test systems are implemented by multi-thread technology, which is relatively simple and can meet the needs of most black-box tests. However, the operating system has constraints on the size of the stack space for thread technology, and when the number of threads reaches a certain scale, the execution efficiency of the system will drop sharply. These limitations lead to the fact that the TTCN-3 operating environment implemented by multi-threading technology cannot be applied to large-scale stress testing.

Contents of the invention

In view of the above-mentioned defects of the prior art, the object of the invention is to provide a TTCN-3 testing system and testing method based on C++ mapping. The test system and test method make full use of the characteristics of C++, which can make the corresponding TTCN-3 operating environment meet the needs of execution efficiency and secondary development for the system under test, and avoid the relevant restrictions brought by thread technology.

For realizing above-mentioned purpose of the invention, the present invention adopts following technical scheme:

A kind of TTCN-3 test system based on C++ mapping is characterized in that:

The system includes a test manager, a language converter, a test case executor, a shared memory manager, an internal encoder/decoder, a C++ mapping interface, a system under test adapter, and an operating system adapter, wherein

The test manager manages the entire test system; the language converter converts the TTCN-3 test case, and generates a semantically equivalent C++ test case, which is executed by the test case executor;

The test component in the test case encodes and decodes the data stored in the shared memory by the internal encoder/decoder, and the shared memory manager manages the shared memory;

The C++ mapping interface includes a test control interface and a test execution interface, the test control interface realizes the connection between the test case executor and the system adapter under test and the operating system adapter, and the test execution interface realizes A connection between the test manager and the test case executor.

Wherein, the test case executor includes the TTCN_Proces class, the TTCN_Process_Options class and the TTCN_Process_Manager class, wherein the class TTCN_Proces realizes the encapsulation of the test component/process; the class TTCN_Process_Options realizes the encapsulation of the test component/process-related attributes; the class TTCN_Process_Manager is used to manage the test components.

The shared memory manager opens a three-layer interface to the application program, including the container library conforming to the STL specification; the new and delete interfaces conforming to the C++ specification; and the allocate and deallocate interfaces conforming to the C specification.

Each test component in the test case is implemented as a process.

The inter-process communication technology conforming to the POSIX standard is adopted among the processes.

The test system adopts a layered structure, and the function call interface between layers strictly meets the C++ mapping standard.

The inner encoder/decoder uses reflection mechanism to dynamically identify the type and size of data.

A kind of test method realized based on above-mentioned TTCN-3 test system, it is characterized in that comprising the steps:

(1) For the abstract test suite of TTCN-3, it is converted into a C++ test case suite by the language converter;

(2) described C++ test case cover is together with operating environment library, is linked in C++ compiler, generates test case library file and loads and executes for test case executor;

(3) The test system starts the background process, initializes the environment, and loads the configuration file;

(4) According to the test case to be executed, load the corresponding test case library file;

(5) Start the test management process and instantiate the test cases in the library file;

(6) Execute test cases and report test execution results.

Wherein, the language converter performs the following processing on the abstract test suite of TTCN-3: after the lexical analyzer and the syntax analyzer, an abstract syntax tree is generated, and the semantic information extractor extracts the abstract syntax tree from the abstract syntax tree while doing semantic inspection. Extract the semantic information from , fill in each manager, including symbol manager, type manager, converter, global information index and scope manager, and finally the code generator will process the information of each manager, in the helper class With help, generate test cases that satisfy the C++ mapping interface.

The language converter adds type attribute information to the data in the generated C++ test case.

The TTCN-3 test system and test method based on C++ mapping implemented by the present invention can not only adapt to the requirements of general systems for black-box testing, but also meet the actual needs of testing systems with concurrent, interactive, and real-time characteristics. Wider range.

Description of drawings

The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

Figure 1 is a schematic diagram of the basic execution flow of the language converter.

Figure 2 is a schematic diagram of the main class structure relationship of the test case executor.

FIG. 3 is a schematic diagram of a basic hierarchical structure of a shared memory manager.

FIG. 4 is a schematic diagram of a procedure call between two processes based on shared memory.

Fig. 5 is a schematic diagram of the basic structure of the test control interface oriented to the test manager.

FIG. 6 is a schematic diagram of the basic structure of the test execution interface and the test control interface oriented to the test executor.

Fig. 7 is a flow chart of the TTCN-3 testing method based on C++ mapping according to the present invention.

Detailed ways

As we all know, C is not an object-oriented programming language, and it does not have the feature of reuse. It is difficult to directly transplant the program compiled for a specific environment to another environment, so the development efficiency of using C language is not high. C++, like Java, is an object-oriented programming language, and they can be reused through technical means such as encapsulation, inheritance, and polymorphism. However, the compiled code of the Java language is a kind of byte code, which is interpreted and executed by the Java virtual machine on the operating system at runtime, so the operating efficiency is far inferior to that of C++. Compared with C, C++ has object-oriented features, which is convenient for secondary development of the specific system under test; on the other hand, it has higher operating efficiency just like C language. So using C++ to realize TTCN-3 operating environment is an ideal choice at present.

In order to use C++ to realize the TTCN-3 operating environment, the work we need to do includes establishing the corresponding C++ mapping interface. The mapping interface includes a test control interface and a test execution interface. In addition, functional modules including test manager, language converter, test executor, shared memory manager, internal encoder/decoder, system under test adapter and operating system adapter need to be established. The above-mentioned collection of these functional modules is the TTCN-3 test system based on C++ mapping in the present invention. These functional modules are described separately below.

The function of the test manager is to be responsible for the management of the entire TTCN-3 test system, and its main tasks include:

◆Test control: After the system is initialized, all tests are executed from the test control service, which is responsible for loading the test suite and providing the corresponding test suite startup parameters.

◆Log management: responsible for maintaining the system log, and responsible for recording the content of various event notifications sent to it by other modules during the test execution process.

◆Test component management: In many cases, the execution of a test case needs to be completed by multiple test components, and these components may be distributed in different physical systems. The test component management function is responsible for the creation, distribution and coordinated execution of each test component.

◆Error handling: When errors or exceptions occur during test execution, error (abnormal) recovery and error (abnormal) cleaning are performed.

The above-mentioned test suite is a unique module of the TTCN-3 system, which explicitly or implicitly describes all definitions and behavior descriptions required to completely define a set of test cases through import statements.

In the actual TTCN-3 system operating environment, the test manager is displayed as a graphical user interface, and its menu structure includes the following contents:

● Configuration: various parameters of the system and the configuration of the adapter.

◆System parameter configuration: realize the relevant system configuration of the test system by reading the system initialization file;

◆Adapter Settings: Set the adapter file used by the current test case.

● Execution: implement the execution flow of test cases.

◆Module loading: load the modules compiled into dynamic (or shared library);

◆Test module specification: specify the module name used for the current test task in the loaded module list;

◆Module parameter setting: support dynamic setting of module parameters, allowing testers to set the corresponding parameters of the module on the GUI interface;

◆Specify test case execution: After the module is loaded and the test module is specified, all test cases in the current module will be automatically listed on the menu item, and the user can specify the test case used in the current test task;

◆Execute test: start to execute the test;

◆Execution cancel: cancel the test task currently being executed.

●Statistics: reflect the current test execution status, and count the number of PTCs such as normal end and failure. The statistics of the following indicators are mainly realized:

◆The total number of use cases of the current test task;

◆The total number of test cases currently passed;

◆Currently running PTC data;

◆PTC peak value of the current test task;

◆The number of PTCs passed by the current test case;

◆PTC data that the current test case fails;

◆PTC peak value of the current test case;

◆The total number of processes in the test system;

◆The number of shared memory requested by the system;

◆The number of shared memory currently used;

◆Peak value of memory used;

●Log query: the system supports the query of the test log, and this item realizes re-outputting the log written in the test log file to the console.

●Online help: Realize the online help function of the test system.

The role of the language converter is to convert TTCN-3 test cases and generate semantically equivalent C++ test cases, and these converted test cases meet the C++ mapping interface standard. It is one of the core technologies of the present invention.

Figure 1 shows the basic execution flow of the language converter. The TTCN-3 test suite generates an abstract syntax tree with correct syntax after passing through the lexical analyzer and syntax analyzer, but the semantics are not necessarily correct at this time, and the semantic information needs to be extracted from the abstract syntax tree for inspection; in addition, when converting During the process, the information needed in many places may be provided later. Usually, the backfill mechanism adopted is relatively simple, but it is not flexible enough, especially when there is a lot of information to be backfilled, and we do not know its structure. Therefore, while doing semantic check, the semantic information extractor extracts semantic information from the abstract syntax tree and fills in various managers, including symbol manager, type manager, converter, global information index and scope manager. Finally, the information of each manager is processed by the code generator, and with the help of the helper class, a test case that satisfies the C++ mapping interface is generated.

The language converter is implemented using compiling technology. This is a subject that can be easily solved by those skilled in the art, so it will not be described in detail here.

The test case executor is responsible for executing test cases written in a standard test language and provides various basic services, including:

◆Test configuration: A test configuration consists of a group of components that are associated with each other through communication ports. This service can implement dynamic configuration of test components.

◆Selection of use cases: A test suite often contains a large number of test cases. The use case selection service determines whether each test case is executed and the number of execution times according to the test strategy.

◆Execution engine: responsible for compiling and executing the standard test language.

◆Template and matching: Provide matching service between data, used for data detection. This service can be used to accept and send some specific data sets.

◆Clock management: Provide basic clock management functions, including start, read, write, stop, trigger, etc.; meanwhile, hide the clock implementation details of different operating systems to provide a unified clock service interface.

◆Test component communication: Provide message or process-based component communication services to facilitate the cooperation between test components to jointly complete the execution of a test case.

◆Parameterization: Various forms of parameterization are performed on various data in the test case during the test operation.

◆Evaluation of Execution Results: Evaluate the execution results of test cases, which introduces a two-level evaluation mechanism, that is, local result evaluation and global result evaluation, and specifies 4 possible values and related value coverage rules.

Figure 2 depicts the main class structure relationships of the test case executor. Because test components are all implemented as processes, the functions and structures of these classes are closely related to the management methods and functions of processes. The class TTCN_Proces implements the encapsulation of test components/processes; the class TTCN_Process_Options implements the encapsulation of test components/process-related attributes; the class TTCN_Process_Manager is mainly used to manage test components.

The basic hierarchy of the shared memory manager is shown in Figure 3. Among them, the role of the ShmInitializer class is to initialize the entire shared memory area. It directly calls shmget, shmat and other functions to apply for shared memory space, and guarantees the size of shared memory and the uniqueness of multi-process applications are all the work of this class. The SharedMemory class is responsible for direct management of shared memory. It provides C-standard shared memory operation interfaces such as Allocate and Deallocate for applications, and shared memory object operation interfaces such as AddObject and GetObject. The latter enables inter-processes to use the shared memory manager. Communication becomes possible. The ShmObject class uses the interface of SharedMemory and encapsulates its functions such as Allocate, and provides C++ standard memory operation interfaces such as new and delete for applications. As long as any class inherits from ShmObject, it can directly use new and delete operations. Based on the ShmObject class, a simple STL type container library can be developed. Among them, List, Queue, and Priority Queue are three relatively complete containers, and their interfaces basically conform to the STL specification. In addition, in Figure 3, the shared memory manager opens three-layer interfaces to the application program, including the above-mentioned container library conforming to the STL specification, the new and delete interfaces of the C++ specification, and the allocate and deallocate interfaces of the C specification. In practice, The application program can completely choose one or more layers of interfaces to use according to the needs.

When testing, each test component in the test case, including the main test component, parallel test component, and system test component, is implemented as a process. Compared with the thread implementation method, it avoids the constraint of the size of the stack space. In addition, it will not cause a sharp drop in system execution efficiency due to the rapid increase in the number of test components. The creation of a process corresponds to the create action of the test component. In terms of specific code implementation, first call the system call fork () to create a child process, and then create an instance of the parallel test component or the main test component. In order to facilitate and quickly exchange data between processes, the system adopts POSIX (Portable Operating System Interface) inter-process communication technology, including shared memory, mutex lock, event, etc., among which shared memory is used to store communication data, and event is used to In order to notify the change of data in the shared memory, the mutex is used for the synchronous access of each process to the shared memory.

The process of making a procedure call between two processes based on shared memory is shown in FIG. 4 . The process PTC1 writes the call application to the shared memory after being encoded by the encoder/decoder through the port Port1. The read event thread of the process PTC2 reads the procedure call event from the shared memory and uses the encoder/decoder to decode it. After decoding, it is found that The procedure call application belonging to port Port2 is distributed to port Port2 on PTC2, and Port2 executes the called procedure, and writes the execution result code into the shared memory; the read event thread of PTC1 reads the returned result from the shared memory And decode it, distribute it to port Port1 on PTC1.

The role of the internal encoder/decoder is to add and decode data expressed in C++. A test component must use the internal encoder/decoder to encode the data before storing it in the shared memory; another test component needs to use the internal encoder/decoder to decode the data read from the shared memory before it can be used. For coding, we implement a set of simplified reflection mechanism for C++ data objects, which can support: 1) instantiation of objects by name; 2) traversal of variables in objects without knowing the object type , to assign and retrieve values for variables. The specific implementation method of this mechanism is introduced below.

1. Realization of instantiation by name:

1. All classes must inherit from the base class TTCNValue, so that all objects can be referenced through TTCNValue pointers.

2. In each class that needs to implement reflection, a TTCNValue*newInstance() method needs to be provided, similar to the factory pattern, in which a pointer to a specific object will be created.

3. Create a static map pointer in the static area of the program to save the mapping between the class name and type information, so as to realize instantiation by name. As mentioned earlier, all the information of the class is saved in the created object, so the map saves the class name and the pointer of the object of the class. This map is created when the program starts.

4. When the name of a class is given, we go to the mapping of the map to find it. If we find the name of the class, we call the newInstance() method of the corresponding object, thus creating a new instance of the class. object. It also realizes the instantiation of the object by name.

2. Traversal of variables in objects by name

1. The TTCNType class is used to save the variable information of the class, and the class TTCNValue has a static protected member variable pointing to TTCNType*. In this way, each class inherited from TTCNValue has a unique TTCNType object to report its type information.

2. The TTCNType corresponding to each class stores the name of the class.

3. TTCNValue has a vector<TTCNValue*>values that holds pointers to member variables. In the constructor of each concrete type, the pointers to its member variables are stored in the vector in sequence.

4. The information (variable name, type name) of the member variables of the class is stored in the simple class Attribute{attrName, typeName}.

5. In TTCNType, there is a vector<Attribute>attributes that saves member variable attributes (Attribute). In the constructor of each concrete type, the properties of its member variables are stored in the vector in order.

TTCNValue provides the methods TTCNValue &getAttributeByName(stringname), setAttributeValue(string name, TTCNValue &value), and you can operate on the value of the member variable by the name of the variable.

Using this reflection mechanism, the internal encoder/decoder can dynamically identify the type and size of data. In order to be able to use this reflection mechanism at execution time, the language converter also needs to add various type attribute information for various data in the generated C++ test case.

The role of the system under test adapter is: the test executor needs to communicate with the system under test continuously based on message and process to complete the interactive operation during the test case running, and these interactive operations must be carried out by the system adapter to the specific system under test. The adaptation can be completed. In addition, in order to be able to communicate, the adapter of the system under test needs to convert the corresponding format of the transmitted data.

The function of the operating system adapter is that during the execution of the test case, the test executor needs to call various external functions. The operating system adapter adapts to different operating systems and hides the details of different operating systems from the test executor. In addition, operating system adapters can provide unified runtime support in addition to facilitating the use of external functions by the test executor.

Figure 5 shows the basic structure of the test control interface facing the test manager, and Figure 6 shows the basic structure of the test execution interface and the test control interface facing the test executive. As shown in Figure 5 and Figure 6, TriCommunicationSA, TriPlatformSA and TriPlatformPA constitute the test execution interface, the first two are the call interface between the test case executor and the system adapter under test, the third is the test case executor and the operation Call interface between system adapters. The test execution interface is a standard interface for the interaction between the TTCN-3 operating environment and the external system, and the system adapter under test and the operating system adapter are all developed based on it. TciTMRequired, TciTMProvided, TciCHRequired and TciCHProvided together constitute the test control interface, where the first two interface classes describe the methods that the test manager is called and should provide, and the latter two interface classes describe the methods that the test case executor is called and should provide method. The test control interface is the interface between the test manager and the test case executor, and it is mainly used for message passing between the test manager and the test case executor during test case execution. The test case executor is started by the test manager. During the execution process, the test case executor needs to continuously report the current execution status to the test manager. The test manager reports the current test case execution status to the user according to the execution status information, and Final test results.

The TTCN-3 test system composed of the above modules adopts a layered structure, and the function call interface between each layer strictly meets the C++ mapping standard.

Using the above-mentioned functional modules, the specific method for performing the test is shown in Figure 7, including the following steps:

(1) For the abstract test suite (ATS) of TTCN-3, it is converted into a C++ test case suite by a language converter;

(2) The above-mentioned C++ test case cover is combined with the runtime environment library RBLib.a in the C++ compiler to generate the test case library file TSlib.a for the test case executor to load and execute;

(3) The test system starts the background process, initializes the environment, and loads the configuration file;

(4) According to the test case to be executed, load the corresponding test case library file TSlib.a;

(5) Start the test management process and instantiate the test case testcase in the library file;

(6) Execute the test case testcase and report the test execution result.

In Figure 7, the right side is the main flow of the TTCN-3 operating environment, and the left side is the flow executed by the language converter. Through the above steps, by formulating the C++ mapping interface standard, the test case described in TTCN-3 can be converted into an equivalent test case described in C++ by the language converter, and the operating environment compiles, loads, and executes the converted C++ Test cases can make full use of the characteristics of C++ to achieve high efficiency in the testing process and meet the requirements for testing systems with concurrency, interaction, and real-time characteristics.

The C++ mapping-based TTCN-3 test system and test method of the present invention have been described in detail above, but obviously the specific implementation form of the present invention is not limited thereto. For those skilled in the art, various obvious changes made to the method of the present invention without departing from the spirit of the method and the scope of the claims are within the protection scope of the present invention.

Claims (10)

1. a kind of TTCN-3 test system based on C++ mapping, is characterized in that: The system includes a test manager, a language converter, a test case executor, a shared memory manager, an internal encoder/decoder, a C++ mapping interface, a system under test adapter, and an operating system adapter, wherein The test manager manages the entire test system; the language converter converts the TTCN-3 test case, and generates a semantically equivalent C++ test case, which is executed by the test case executor; The test component in the test case encodes and decodes the data stored in the shared memory by the internal encoder/decoder, and the shared memory manager manages the shared memory; The C++ mapping interface includes a test control interface and a test execution interface, the test control interface realizes the connection between the test case executor and the system adapter under test and the operating system adapter, and the test execution interface realizes A connection between the test manager and the test case executor. 2. the TTCN-3 test system based on C++ mapping as claimed in claim 1, is characterized in that: Described test case executor comprises TTCN_Proces class, TTCN_Process_Options class and TTCN_Process_Manager class, wherein said class TTCN_Proces realizes the encapsulation to test component/process; Class TTCN_Process_Options realizes the encapsulation of test component/process related attribute; Class TTCN_Process_Manager is used for managing test component. 3. the TTCN-3 test system based on C++ mapping as claimed in claim, is characterized in that: The shared memory manager opens a three-layer interface to the application program, including the container library conforming to the STL specification; the new and delete interfaces conforming to the C++ specification; and the allocate and deallocate interfaces conforming to the C specification. 4. the TTCN-3 test system based on C++ mapping as claimed in claim 1, is characterized in that: Each test component in the test case is implemented as a process. 5. the TTCN-3 test system based on C++ mapping as claimed in claim 4, is characterized in that: The inter-process communication technology conforming to the POSIX standard is adopted among the processes. 6. the TTCN-3 test system based on C++ mapping as claimed in claim 1, is characterized in that: The test system adopts a layered structure, and the function call interface between layers strictly meets the C++ mapping standard. 7. the TTCN-3 test system based on C++ mapping as claimed in claim 1, is characterized in that: The inner encoder/decoder uses reflection mechanism to dynamically identify the type and size of data. 8. A test method realized based on TTCN-3 test system as claimed in claim 1, is characterized in that comprising the steps: (1) For the abstract test suite of TTCN-3, it is converted into a C++ test case suite by the language converter; (2) described C++ test case cover is together with operating environment library, is linked in C++ compiler, generates test case library file and loads and executes for test case executor; (3) The test system starts the background process, initializes the environment, and loads the configuration file; (4) According to the test case to be executed, load the corresponding test case library file; (5) Start the test management process and instantiate the test cases in the library file; (6) Execute test cases and report test execution results. 9. testing method as claimed in claim 8, is characterized in that: The language converter performs the following processing on the abstract test suite of TTCN-3: after the lexical analyzer and the syntax analyzer, an abstract syntax tree is generated, and the semantic information extractor extracts from the abstract syntax tree while performing semantic inspection. Output semantic information, fill in each manager, including symbol manager, type manager, converter, global information index and scope manager, and finally the code generator will process the information of each manager, with the help of helper class , to generate test cases that satisfy the C++ mapping interface. 10. testing method as claimed in claim 9, is characterized in that: The language converter adds type attribute information to the data in the generated C++ test case.

Images