CN114817047A

CN114817047A - Compiler test method, case generation method and device and instruction storage structure

Info

Publication number: CN114817047A
Application number: CN202210474322.4A
Authority: CN
Inventors: 苗瑞秋; 侯化成; 王野; 徐宁仪
Original assignee: Shanghai Power Tensors Intelligent Technology Co Ltd
Current assignee: Shanghai Power Tensors Intelligent Technology Co Ltd
Priority date: 2022-04-29
Filing date: 2022-04-29
Publication date: 2022-07-29
Also published as: WO2023207973A1

Abstract

The embodiment of the disclosure provides a test method, a test device, test equipment, a storage medium and a test instruction storage structure of a compiler. The method comprises the following steps: constructing a test case based on data related to the test instruction stored in an instruction database and a reference compiling instruction corresponding to the test case; compiling the test case by using a compiler to be tested to obtain a target compiling instruction; and judging whether the compiler has abnormity or not based on a comparison result of the reference compiling instruction corresponding to the test case and the target compiling instruction. The embodiment of the disclosure can realize the automatic test of the compiler, and improve the test efficiency and the accuracy of the test result.

Description

Compiler test method, case generation method and device and instruction storage structure

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a test method, a case generation method, an apparatus, a device, a storage medium, and a storage structure of a test instruction for a compiler.

Background

A compiler is generally used to compile a program from one language to another language, and the accuracy of the compiled result of the compiler is a prerequisite for ensuring the normal operation of the program. If the function of the compiler is abnormal, the program cannot obtain the originally expected running result, and even debug of the subsequent program is misled. Therefore, after developing a compiler, it is necessary to provide a solution that can test the compiler and discover the exception of the compiler.

Disclosure of Invention

The disclosure provides a test method of a compiler, a case generation method, a case generation device, equipment, a storage medium and a storage structure of a test instruction.

According to a first aspect of the embodiments of the present disclosure, there is provided a method for testing a compiler, the method including:

constructing a test case based on data related to the test instruction stored in an instruction database and a reference compiling instruction corresponding to the test case;

compiling the test case by using a compiler to be tested to obtain a target compiling instruction;

and judging whether the compiler has abnormity or not based on a comparison result of the reference compiling instruction corresponding to the test case and the target compiling instruction.

In some embodiments, the test cases include functional test cases and/or stability test cases, each of the functional test cases includes a single test instruction for testing the function of the compiler; and each stability test case is obtained by splicing a plurality of test instructions and is used for testing the stability of the compiler.

In some embodiments, the stability test case meets one or more of the following conditions:

the time length for compiling the stability test case by the compiler is longer than the preset time length;

the number of test instructions included in the stability test case is larger than a preset number;

and the data volume of the stability test case is larger than the preset data volume.

In some embodiments, the determining whether the compiler has an exception based on a comparison result between a reference compilation instruction corresponding to the test case and the target compilation instruction includes:

judging whether the function of the compiler is abnormal or not based on a comparison result of the reference compiling instruction corresponding to each functional test case and the target compiling instruction corresponding to each functional test case;

and when the function of the compiler is not abnormal, judging whether the stability of the compiler is abnormal or not based on a comparison result of a reference compiling instruction corresponding to the stability test case and a target compiling instruction corresponding to the stability test case.

In some embodiments, the test instruction includes a plurality of components, the data related to the test instruction includes a plurality of pieces of feature data corresponding to any component and reference compiled data corresponding to any piece of feature data, and the feature data corresponding to any component is stored in association with the feature data corresponding to the rest of the components except for the any component.

In some embodiments, the component parts include one or more of: the test instruction comprises an instruction type, a control domain and an instruction address, wherein the control domain is used for representing a control operation included by the test instruction.

In some embodiments, the test case includes a plurality of functional test cases for testing the compiler function, the test case is constructed based on data related to the test instruction stored in an instruction database, and a reference compiling instruction corresponding to the test case includes:

traversing each piece of feature data corresponding to the plurality of components, and respectively determining associated feature data stored in association with the target feature data from the rest of the components except any component in the plurality of components aiming at any item mark feature data corresponding to any component;

generating a test instruction based on the target characteristic data and the associated characteristic data, and obtaining a reference compiling instruction corresponding to the test instruction based on reference compiling data corresponding to the target characteristic data and reference compiling data corresponding to the associated characteristic data;

and respectively taking each generated test instruction as a functional test case, and taking a reference compiling instruction corresponding to each test instruction as a reference compiling instruction corresponding to the functional test case.

In some embodiments, the test cases include stability test cases for testing stability of the compiler, the test cases are constructed based on data related to test instructions stored in an instruction database, and reference compiling instructions corresponding to the test cases include:

for any item of labeled feature data in any component, determining associated feature data stored in association with the target feature data from the rest of the components except for any component in the plurality of components respectively;

and splicing the generated multiple test instructions to obtain the stability test case, and splicing the reference compiling instructions corresponding to the multiple test instructions to obtain the reference compiling instructions corresponding to the stability test case.

In some embodiments, the data related to the test instruction is stored through a tree structure, a root node of any tree structure is used for storing a piece of target feature data in the instruction type and reference compiled data corresponding to the target feature data, and child nodes to leaf nodes of the tree structure are used for sequentially storing first associated feature data related to the target feature data in the control domain, reference compiled data corresponding to the first associated feature data, and second associated feature data related to the target feature data and the first associated feature data in the instruction address.

In some embodiments, the constructing a test case based on data related to the test instruction stored in the instruction database, and the reference compiling instruction corresponding to the test case include:

acquiring feature data stored by each node in any path from a root node to a leaf node in the tree structure and reference compiling data corresponding to the feature data;

splicing the characteristic data stored in each node to obtain a test instruction, and splicing the reference compiling data corresponding to the characteristic data stored in each node to obtain a reference compiling instruction corresponding to the test instruction;

and constructing a test case based on the obtained test instruction, and constructing a reference compiling instruction corresponding to the test case based on the reference compiling instruction corresponding to the test instruction.

In some embodiments, after determining that there is an exception in the compiler based on a comparison result between a reference compiled instruction corresponding to the test case and the target compiled instruction, the method further includes:

outputting a test report, the test report including one or more of the following information: the type of the exception, the test instruction with the exception, and the component with the exception in the test instruction;

the type of the exception comprises a functional exception or a stability exception, and the component comprises an instruction type, a control domain and an instruction address.

According to a second aspect of the embodiments of the present disclosure, there is provided a storage structure of test instructions, the storage structure comprising: a tree structure storage structure;

the tree structure is used for storing data related to the test instruction; the data related to the test instruction comprises a plurality of pieces of feature data corresponding to each of a plurality of components of the test instruction, and reference compiling data corresponding to each piece of feature data; the plurality of components comprise an instruction type, a control domain and an instruction address, wherein the control domain represents a control operation included by the test instruction;

the root node of the tree structure is used for storing a piece of target feature data in the instruction type and reference compiled data corresponding to the target feature data, and the child nodes of the tree structure to the leaf nodes of the tree structure are used for sequentially storing first associated feature data associated with the target feature data in the control domain, reference compiled data corresponding to the first associated feature data, and second associated feature data associated with the target feature data and the first associated feature data in the instruction address.

According to a third aspect of the embodiments of the present disclosure, there is provided a method for generating a test case, where the test case is used for testing a compiler, and a test instruction for constructing the test case is stored by using the storage structure according to the second aspect, the method includes:

splicing the feature data stored in each node to obtain a test instruction, and splicing the reference compiling data corresponding to each feature data to obtain a reference compiling instruction corresponding to the test instruction;

and generating a test case based on the obtained test instruction, and generating a reference compiling instruction corresponding to the test case based on the reference compiling instruction corresponding to the test instruction.

According to a fifth aspect of embodiments of the present disclosure, there is provided an electronic device, the electronic device including a processor, a memory, and computer instructions stored in the memory and executable by the processor, wherein the processor executes the computer instructions to implement the method according to the first aspect and/or the third aspect.

According to a sixth aspect of embodiments of the present disclosure, there is provided a computer-readable storage medium having stored thereon computer instructions which, when executed, implement the method of the first and/or third aspect mentioned above.

In the embodiment of the disclosure, when a compiler is tested, an instruction database may be preset, data related to a test instruction may be stored in the instruction database, then a test case and a reference compiler instruction corresponding to the test case may be automatically constructed based on the data related to the test instruction stored in the instruction database, the test case is compiled by using the compiler to be tested, a target compilation instruction is obtained, and whether the compiler is abnormal or not is determined based on a comparison result of the target compilation instruction and the reference compiler instruction corresponding to the test case. By the method, the test case can be automatically constructed based on the data stored in the instruction database, the compiler can be automatically tested, the automation of the whole test process can be realized, and the test efficiency and the accuracy of the test result are improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure.

Fig. 1(a) is a flowchart of a testing method of a compiler according to an embodiment of the present disclosure.

Fig. 1(b) is a schematic structural diagram of a test instruction according to an embodiment of the present disclosure.

Fig. 1(c) is a diagram of a tree structure storing data related to a test instruction according to an embodiment of the disclosure.

Fig. 1(d) is a schematic path diagram in a tree structure diagram for storing data related to a test instruction according to an embodiment of the disclosure.

FIG. 2 is a schematic diagram of an instruction database according to an embodiment of the disclosure.

FIG. 3 is a diagram illustrating a test instruction storing method according to an embodiment of the disclosure.

Fig. 4 is a schematic logical structure diagram of a testing apparatus of a compiler according to an embodiment of the present disclosure.

Fig. 5 is a schematic diagram of a logical structure of an apparatus according to an embodiment of the present disclosure.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

The terminology used in the present disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used in this disclosure and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality.

It is to be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present disclosure. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.

In order to make the technical solutions in the embodiments of the present disclosure better understood and make the above objects, features and advantages of the embodiments of the present disclosure more comprehensible, the technical solutions in the embodiments of the present disclosure are described in further detail below with reference to the accompanying drawings.

A compiler is usually used to compile a program from one language into another language, and the accuracy of the compilation result of the compiler is a prerequisite for ensuring the normal operation of the program. If the function of the compiler is abnormal, the program cannot obtain the originally expected running result, and even debug of the subsequent program is misled. Taking an assembler as an example, the assembler is generally used to convert an assembler language into a machine language, and if the function of the assembler is abnormal, the finally obtained machine language may be wrong, which may cause an error in the operation result of the final program.

For some self-developed compilers, it is usually necessary to test their functions first and then put them into products for use. In the related art, a few schemes specially for independently testing a compiler are used, and most schemes are used for testing the whole system running a program, that is, the whole system is tested, the final running result and the expected result of the program are compared to determine whether the whole system is abnormal, and if so, whether the compiling link is abnormal is checked. This approach alone does not test the functionality of the compiler. In some techniques, the compiler is directly tested manually, for example, the compiling result of the compiler is compared with the correct result manually, which is tedious, time-consuming, labor-consuming, and prone to error.

Based on this, an embodiment of the present application provides a test method for a compiler, which may preset an instruction database, where the instruction database may store data related to a test instruction, then may automatically construct a test case based on the data related to the test instruction stored in the instruction database, and a reference compiler instruction corresponding to the test case, then compile the test case using the compiler to be tested, to obtain a target compilation instruction, and determine whether the compiler is abnormal based on a comparison result between the target compilation instruction and the reference compiler instruction corresponding to the test case. By the method, the test case can be automatically constructed based on the data stored in the instruction database, the compiler can be automatically tested, the automation of the whole test process can be realized, and the test efficiency and the accuracy of the test result are improved.

The testing method of the compiler in the embodiment of the present disclosure may be executed by various electronic devices, for example, a mobile phone, a computer, a cloud server, and the like.

The compiler in the embodiments of the present disclosure may be software or hardware that converts one language of a program into another language, and for example, the compiler may be a software program having a compiling function or a hardware device having a compiling function. The compiler may be an assembler that converts assembly language into machine language or a compiler with other types of language conversion functionality.

For convenience of distinction, data obtained by correctly compiling the feature data corresponding to each component in the test instruction by the compiler is referred to as reference compilation data, and data obtained by correctly compiling the test instruction or the test case to be compiled by the compiler is referred to as reference compilation instruction.

As shown in fig. 1(a), the testing mode of the compiler may include the following steps:

s102, constructing a test case based on data related to the test instruction and stored in an instruction database, and a reference compiling instruction corresponding to the test case;

in step S102, a test case may be constructed based on data related to the test instruction stored in the instruction database, and a reference compiling instruction corresponding to the test case. The test case may be a single test instruction, or a test document constructed by a large number of test instructions. The data related to the test instruction may be various data that can determine the test instruction, and various data that can determine a reference compiled instruction corresponding to the test instruction. For example, in some scenarios, the data related to the test instruction may be a complete test instruction and a reference compiling instruction corresponding to the test instruction, for example, a plurality of test instructions and a reference compiling instruction corresponding to each test instruction may be directly stored in the instruction data, and then the test case may be directly constructed according to the test instruction. In some scenarios, the test instruction may include a plurality of components, such as an instruction type, a control field, an instruction address, and the like, where the control field is a control operation included in the instruction, such as a rounding operation, a saturation processing operation, and the like, and the data related to the test instruction may be feature data corresponding to each component of the test instruction, that is, specific content of each component (for example, if the component is an instruction type, the feature data may be an addition instruction, a subtraction instruction, and the like), and reference compiled data corresponding to the feature data. For example, the instruction database may also store feature data corresponding to each component and reference compilation data corresponding to the feature data, and then construct a test instruction based on the feature data, and further construct a test case based on the constructed test instruction, and a reference compilation instruction corresponding to the test case.

S104, compiling the test case by using a compiler to be tested to obtain a target compiling instruction;

in step S104, after the test case is obtained, the test case may be compiled by using a compiler to be tested, so as to obtain a target compiling instruction.

S106, judging whether the compiler is abnormal or not based on a comparison result of the reference compiling instruction corresponding to the test case and the target compiling instruction.

In step S106, the target compiling instruction compiled by the compiler may be compared with the reference compiling instruction corresponding to the test case, if the target compiling instruction and the reference compiling instruction are consistent, it indicates that the compiling result of the compiler is correct, and if the target compiling instruction and the reference compiling instruction are not consistent, it indicates that the compiler result of the compiler is wrong and the compiler has an exception.

In some embodiments, the test case may be a test case for testing the functionality of a compiler, hereinafter referred to as a functional test case. Wherein each functional test case may include a single test instruction. Generally, a compiler may be tested by a plurality of functional test cases respectively to detect whether there is an exception in the compiling function of the compiler, for example, whether the compiler wrongly compiles a certain type of instruction, or wrongly compiles a certain control operation in the instruction, or wrongly compiles a certain type of address in the instruction. Generally, in order to more fully test the functions of a compiler, a plurality of functional test cases can be generally constructed, each of which includes a test instruction, and the test instructions can cover various instruction types, control operations and instruction addresses to ensure the comprehensiveness and integrity of test data.

In some embodiments, the test case may be a test case for testing stability of a compiler, hereinafter referred to as a stability test case, and each stability test case may be spliced by a plurality of test instructions. Currently, when testing a compiler, only the compiler's functionality is typically tested. However, even when the compiler functions normally, a scenario exists in which the compiler crashes and cannot work normally due to a large amount of data to be compiled, and therefore, the stability of the compiler can be tested to detect whether an exception exists when the compiler compiles a large number of instructions at one time. When the stability of the compiler is tested, a large number of test instructions can be utilized to splice to obtain one or more stability test cases with larger data volume for testing the stability of the compiler.

In some embodiments, the test cases may include both functional test cases and stability test cases, i.e., test both of the above-mentioned performances of the compiler.

The stability test case is used for testing the stability of the compiler, namely, whether the compiler can crash the program or not under the condition that the compiler processes a large amount of data at one time. Therefore, the data volume of the stability test case is as large as possible, so as to obtain an accurate test result. In some embodiments, the stability test case may comply with one or more of the following conditions: for example, the condition may be that the time length for the compiler to compile the stability test case is longer than a preset time length, so that it can be ensured that no exception occurs after the compiler runs for a long time. The preset duration can be flexibly set based on actual conditions. The condition may also be that the number of the test instructions included in the stability test case may be greater than a preset number, for example, the stability test case needs to be obtained by splicing at least 1000 test instructions. The condition may also be that the data size of the stability test case is greater than a preset data size, for example, the data size of the stability test case is greater than 1 GB. Of course, the above is only an illustrative example, and actually, when constructing the stability test case, the evaluation criterion may also be set to other criteria, as long as it can be ensured that the stability test case has a large data volume, is complex, and can objectively consider the stability of the compiler.

In some embodiments, the built test cases include functional test cases and stability test cases, i.e., the compiler may be subjected to functional tests and stability tests. When the two types of tests are performed on the compiler, the functionality of the compiler can be tested first, and after the condition that the function of the compiler is not abnormal is ensured, the stability of the compiler is further tested. For example, when it is determined whether the compiler has an abnormality based on a comparison result between a reference compilation instruction corresponding to a test case and a target compilation instruction, each functional test case may be obtained one by one, the target compilation instruction corresponding to the functional test case is obtained after the functional test case is compiled by the compiler, then the target compilation instruction is compared with the reference compilation instruction corresponding to the functional test case, and whether the function of the compiler has an abnormality based on the comparison result is determined. Generally, when a compiler is functionally tested, a plurality of functional test cases are used to cover various instruction types, various control operations and instruction addresses. Therefore, each functional test case can be traversed, the above test steps are repeated, and the function of the compiler is tested.

After testing the compiler with all the functionality tests, if the compiler is not found to have an exception, the stability of the compiler is further tested. For example, a compiler may be used to compile each stability test case one by one to obtain a target compilation instruction corresponding to the stability test case, compare the target compilation instruction with a reference compilation instruction corresponding to the stability test case, and determine whether the stability of the compiler is abnormal based on a comparison result.

In general, each test instruction may be made up of a number of components, which in some embodiments may be one or more of an instruction type, a control field, and an instruction address. The instruction address comprises a source address of the test instruction and a target address of the test instruction. For example, as shown in fig. 1(b), the test instruction may include 3 components (components 1-3), where the 3 components correspond to an instruction type, a control field, and an instruction address (a source address and a destination address), respectively. The instruction type may be an addition instruction, a subtraction instruction, a data transfer instruction, an or operation instruction, an and operation instruction, and the like, and the control field indicates a control operation included in the test instruction, and taking the type of arithmetic processing as an example, the control operation included in the test instruction may be a rounding operation, a saturation processing, and the like. The source address of the test instruction may be an address storing data to be processed by the instruction, and the target address of the test instruction may be an address storing data processed by the instruction.

When the test instruction is stored in the instruction database, a plurality of complete test instructions can be directly stored, and when the compiler is tested, each test instruction can be directly obtained. However, such a storage method occupies a larger memory, and it is more complicated to expand or change the test instructions in the instruction database. Therefore, in some embodiments, the data related to the test instruction stored in the instruction database may be a plurality of pieces of feature data corresponding to any component constituting the test instruction, and reference compiled data corresponding to any piece of feature data, for example, the component may be an instruction type, and the feature data may be an addition instruction, a subtraction instruction, or the like. And storing the characteristic data corresponding to each component in association with the characteristic data corresponding to the rest of the components. For example, taking the instruction type as an example, the addition instruction has a control operation corresponding thereto, and the data transmission instruction also has a control operation corresponding thereto, so that the associated feature data existing in each component can be associated, and the test instruction can be conveniently constructed by using the stored associated feature data when constructing the test instruction.

For example, it is assumed that the components of the test instruction include an instruction type, a control field, and an instruction address, where the type of the test instruction corresponds to a plurality of pieces of feature data, each piece of feature data represents an instruction type, and the feature data may be, for example, an addition instruction, a subtraction instruction, or the like. The control field of the test instruction may also correspond to a plurality of pieces of feature data, each piece of feature data representing a control operation, for example, the feature data may be a rounding process, a saturation process, or the like. The instruction address includes a source address of the test instruction and a target address of the test instruction, and the instruction address may also correspond to a plurality of pieces of feature data, each of which represents a group of addresses. Thus, a list of instruction types may be separately stored in the instruction database, the list including a plurality of instruction types, and each instruction type corresponding to the reference compilation data. The control system comprises a control domain list and an instruction address list, wherein the control domain list comprises a plurality of control operations, reference compiling data corresponding to each control operation, and a plurality of groups of addresses, and the reference compiling data corresponding to each group of addresses, wherein each group of addresses comprises a source address and a target address. In this case, the type of control operation involved in each instruction type, for example, an addition instruction, is fixed, and for example, only the control operation related to the addition processing is possible. After the instruction type and the control operation are determined, the type of the data which can be processed is also determined, that is, the data which can be processed is the data in a certain address range, and further, the corresponding instruction address can also be determined. Therefore, when the data is stored in the instruction database, for each instruction type, the instruction type and various control operations, source addresses and target addresses related to the instruction type can be stored in an associated manner, so that when an instruction is constructed based on the characteristic data, the instruction type, the control operations, the source addresses and the target addresses which are associated can be selected based on the association relationship of the characteristic data, and a correct test instruction can be constructed.

In addition, by storing the test instruction in the instruction database by adopting the method, compared with the method of directly storing the complete test instruction, the data size can be reduced, and the method is more convenient and faster when the test instruction stored in the instruction data needs to be changed by adding, deleting, modifying and the like in the follow-up process.

In some embodiments, the instruction database may store feature data corresponding to each component of the test instruction, and reference compiled data corresponding to the feature data. When a functional test case is constructed based on data related to a test instruction stored in an instruction database and a reference compiling instruction corresponding to the functional test case, each piece of feature data corresponding to a plurality of components can be traversed, for any piece of feature data corresponding to any component, hereinafter referred to as target feature data, associated feature data stored in association with the target feature data can be determined from the rest of the components, respectively, then a test instruction is generated based on the target feature data and the associated feature data, and a reference compiling instruction corresponding to the test instruction is obtained based on the reference compiling data corresponding to the target feature data and the reference compiling data corresponding to the associated feature data. After the test instructions are generated, each generated test instruction can be used as a functional test case, and the reference compiling instruction corresponding to each test instruction is used as the reference compiling instruction corresponding to the functional test case, so that a plurality of functional test cases for testing the functions of the compiler can be obtained.

For example, for any item of tag feature data in an instruction type, assuming "add instruction", a plurality of "control operations" and a plurality of "addresses" stored in association with the "add instruction" may be determined from two components, namely, a control field and an instruction address stored in an instruction database, and then a "control operation" and a set of "addresses" may be randomly selected from the plurality of "control operations" and the plurality of "addresses" to construct a test instruction together with the "add instruction".

By traversing each component and each characteristic data in the instruction database, a plurality of test instructions covering various instruction types, control operations and instruction addresses can be constructed, so that the constructed functional test case is more comprehensive, and more accurate test results are obtained.

In some embodiments, the instruction database may store feature data corresponding to each component of the test instruction and reference compilation data corresponding to the feature data, and when a stability test case is constructed based on data related to the test instruction stored in the instruction database and the reference compilation instruction corresponding to the stability test case, associated feature data stored in association with the target feature data may be determined from the remaining components of the plurality of components, respectively, for any item of tagged feature data in any component, and then the test instruction is generated based on the target feature data and the associated feature data. And obtaining a reference compiling instruction corresponding to the test instruction based on the reference compiling data corresponding to the target characteristic data and the reference compiling data corresponding to the associated characteristic data. And then splicing the generated multiple test instructions to obtain a stability test case, and splicing the stability test case to obtain a reference compiling instruction corresponding to the stability test case by using the reference compiling instruction corresponding to the multiple test instructions.

Namely, a plurality of test instructions can be randomly constructed based on the correlated characteristic data stored in the instruction database, and the test instructions are spliced to obtain a stability test case with a large data volume.

In some embodiments, because the components of the test instruction are interrelated, for example, the type of instruction may affect the selection of the control operation, and the type of instruction and the control operation may further affect the selection of the instruction address. I.e., the relationships between the components are interrelated and progressive layer-by-layer. Therefore, when data related to the test instruction is stored in the instruction database, the data may be stored in a tree structure. For any tree structure, the number of layers may be consistent with the number of test instruction components, for example, the test instruction includes three components, i.e., an instruction type, a control domain, and an instruction address, and the tree structure may include three layers, i.e., the tree structure includes a root node, child nodes from which the root node branches, and leaf nodes from which each child node branches, where the root node of the tree structure may be used to store any entry target feature data in the instruction type and reference compilation data corresponding to the target feature data. The sub-nodes of the tree structure to the leaf nodes of the tree structure are used for sequentially storing first associated characteristic data associated with the target characteristic data in the control domain, reference compiling data corresponding to the first associated characteristic data and second associated characteristic data associated with the target characteristic data and the first associated characteristic data in the instruction address.

As shown in fig. 1(c), a schematic diagram of storing a test instruction by using a tree structure is shown, where a root node of the tree structure may be used to store a piece of feature data in an instruction type and its corresponding reference compiled data. For example, the feature data may be "addition instructions" and the reference compiled data may be machine code corresponding to the "addition instructions". For example, there may be three sub-nodes, each sub-node stores a "rounding mode" and its machine code, an "immediate mode" and its machine code, a "register mode" and its machine code, etc., and then for each sub-node, there may be a plurality of leaf nodes, each leaf node stores an instruction address and its machine code associated with the root node and the feature data stored in the sub-node from which the leaf node is branched, for example, a plurality of groups of instruction addresses associated with an "adding instruction", "rounding mode", a "source address 1, a" target address 1 ", a" source address 2, a "target address 2", a "source address 3", a "adding instruction", a "rounding mode", and a "machine code thereof, Destination address 3 ", etc. In some embodiments, if data related to the test instruction is stored in a tree structure, when a test case is constructed, feature data stored in each node in any path from a root node to a leaf node in the tree structure and reference compilation data corresponding to the feature data can be obtained, then the feature data stored in each node are spliced to obtain a test instruction, the reference compilation data corresponding to the feature data stored in each node are spliced to obtain a reference compilation instruction corresponding to the test instruction, then the test case can be constructed based on the obtained test instruction, and the reference compilation instruction corresponding to the test case is constructed based on the reference compilation instruction corresponding to the test instruction. For example, each obtained test instruction can be used as a test case, or a plurality of test instructions can be spliced into a test case.

For example, as shown in fig. 1(d), the "root node-child node-leaf node" framed by the black frame is a path from the root node to the leaf node in the tree structure, and the feature data stored in each node in the path, that is, the "addition instruction", "rounding mode", "source address 1, and target address 1", may be obtained, and a test instruction is obtained by splicing, and the machine codes corresponding to the feature data are obtained, and the reference compiling instruction corresponding to the test instruction is obtained by splicing.

In some embodiments, after determining that the compiler has an exception based on a comparison result between a reference compiling instruction and a target compiling instruction corresponding to the test case, a test report may be further output, where, in order to facilitate a developer to locate the exception, the test report may include one or more of the following information: the exception type comprises a functional exception or a stability exception, and the component parts can be an instruction type, a control domain and an instruction address.

To further illustrate the testing method of the compiler provided by the present disclosure, the following is explained with reference to a specific embodiment.

In the program running process, the corresponding high-level language needs to be compiled into assembly language, then the corresponding assembly language is converted into machine code through an assembler, and then the machine code is sent to a bottom chip for execution. If the compiling result of the compiler is wrong, the program is abnormally operated. The compiling result of the assembler is crucial to the whole program operation, and therefore, the present embodiment provides a testing method of the assembler.

The types of instructions involved in the assembler are complex, so that the assembler can be automatically tested more accurately and comprehensively. An instruction database may be preset, the architecture of the instruction database is shown in fig. 2, because the assembly instruction may be generally divided into three components, i.e., an instruction type, a control domain, and an instruction address, where the instruction type may correspond to an instruction type list, the list includes various instruction types of the assembly instruction and machine codes corresponding to the various instruction types, and similarly, the control domain may also correspond to a control domain list, the list includes various control operations of the assembly instruction and machine codes corresponding to the various control operations, and the instruction address may also correspond to an address list, and the list includes a source address and a destination address of data related to the instruction and machine codes corresponding to the respective addresses. Wherein each piece of feature data in each of the three lists is stored in association with one or more pieces of feature data in the other lists. For example, the instruction type such as "add instruction" in fig. 2 is associated with control operation "round up" and "saturation processing" and is stored in association with "source address 1, destination address 1", "source address 2, destination address 2", "source address 3 and destination address 3" in the instruction addresses, so that when constructing the test instruction, one control operation "saturation processing" and one group of instruction addresses "source address 1 and destination address 1" can be randomly selected to construct one test instruction.

When testing the assembler, for any one of three components of an instruction type, an instruction control domain and an instruction address, traversing each feature data in the component, for each feature data, determining associated feature data of the feature data from the feature data of the other two components, and then randomly selecting one from the associated feature data corresponding to each of the other two components to construct a test instruction. Then, the characteristic data of each component part can be traversed, and the steps are repeated, so that a plurality of test instructions can be obtained, and each test instruction corresponds to the reference machine code.

Then, compiling each test instruction one by using an assembler to be tested to obtain a target compiling instruction, comparing the reference compiling instruction with the target compiling instruction, and determining whether the function of the assembler is abnormal or not based on the comparison result.

If the function of the assembler is not abnormal, a test document can be obtained by splicing a plurality of test instructions, wherein the number of the test instructions in the test document is larger than the preset number. The test document may then be used to test the stability of the assembler to determine if there is an anomaly with the assembler.

If the assembler becomes abnormal, an abnormal test report can be output, and the following information can be indicated in the report: the abnormity is the abnormity of which type, the abnormal test instruction occurs, and which component of the test instruction has the abnormity, so that the research and development personnel can conveniently locate the abnormity and solve the abnormity.

It should be understood that the solutions described in the above embodiments may be combined without conflict, and are not exemplified in the embodiments of the present disclosure.

Typically, a large number of test instructions are required to test a compiler. The test instruction generally consists of a plurality of components, for example, the test instruction can generally consist of three components of an instruction type, a control domain and an instruction address. In the storage process, in order to reduce the amount of stored data and facilitate the change of the stored test instruction, the feature data corresponding to each component of the test instruction and the reference compilation data corresponding to the feature data can be stored separately, and the associated feature data in different components can be stored in an associated manner, so that when the test instruction is generated, the associated feature data in each component can be obtained, the test instruction can be obtained by splicing, and the reference compilation instruction corresponding to the test instruction can be generated, and the test instruction can be more comprehensive and cover various types.

For example, the instruction type may affect the selection of the control operation, and the instruction type and the control operation may further affect the selection of the instruction address, that is, the relationships between the components are related and are progressive layer by layer. In order to avoid splicing components that are not connected when generating a test instruction based on stored components, and obtain an erroneous test instruction, the following provides a method for storing a test instruction, as shown in fig. 3 specifically, the method may include the following steps:

step 1: acquiring data related to a test instruction to be stored, wherein the data related to the test instruction comprises a plurality of pieces of feature data corresponding to each component in a plurality of components of the test instruction and reference compiling data corresponding to each piece of feature data; the plurality of components comprise an instruction type, a control domain and an instruction address, wherein the control domain represents a control operation included by the test instruction;

for example, as shown in fig. 3, the test instruction includes three components, i.e., an instruction type, a control field, and an instruction address, each of which may include a plurality of pieces of feature data, and each piece of feature data has corresponding reference compiled data. For example, the instruction types may include addition instructions, subtraction instructions, data transfer instructions, and the like.

Step 2: and storing the data related to the test instruction through a tree structure, wherein a root node of any tree structure is used for storing a piece of target characteristic data in the instruction type and reference compiled data corresponding to the target characteristic data, and child nodes to leaf nodes of the tree structure are used for sequentially storing first associated characteristic data related to the target characteristic data in the control domain, reference compiled data corresponding to the first associated characteristic data, and second associated characteristic data related to the target characteristic data and the first associated characteristic data in the instruction address.

For example, as shown in fig. 3, the tree structure may include three layers of root node-child node-leaf node, and the signature data a1 (e.g., addition instruction) and its reference compiled data for any entry in the instruction type may be stored in the root node of a tree structure, and then the first associated signature data B1, B2, B3 (e.g., rounding, saturation processing, immediate mode, etc.) associated with the target signature data a1 in the control domain and its respective reference compiled data are determined and stored in each child node from which the root node branches. Then, for the leaf node from which each child node branches, the target feature data a1 stored in the root node and the second associated feature data associated with the first feature data stored in the child node branching off from the leaf node, such as the second associated feature data C1, C2, C3 (e.g., three sets of addresses) associated with the target feature data a1 and the first associated feature data B1, and their respective reference compiled data, may be determined from the instruction address and then stored in each leaf node branching off from the child node where the first associated feature data B1 is located. That is, the feature data stored in each node in any path from the root node to the leaf node in the tree structure may constitute a test instruction.

The specific implementation details of the test instruction storage method may refer to the description of the embodiment in the compiling method, and are not repeated herein.

The present disclosure also provides a storage structure of a test instruction, the storage structure including: a tree structure storage structure;

The specific implementation details of the test instruction storage structure may refer to the description of the embodiment in the compiling method, and are not repeated herein.

In addition, the application also provides a test case generation method, which is used for generating the test case of the test compiler, wherein the test instruction for constructing the test case can be stored by adopting the storage structure. The specific generation process of the test case may include the following steps:

acquiring feature data stored by each node in any path from a root node to a leaf node in a tree structure and reference compiling data corresponding to the feature data;

The process of generating the test case may refer to the description in each embodiment of the compiling method, and is not described herein again.

By the method, the test case for testing the compiler can be automatically generated, errors of the test instruction in the test case can be avoided, and the efficiency and accuracy for testing the compiler are improved. Correspondingly, an embodiment of the present disclosure further provides a testing apparatus for a compiler, as shown in fig. 4, the apparatus includes:

a building module 41, configured to build a test case based on data related to a test instruction stored in an instruction database, and a reference compiling instruction corresponding to the test case;

the compiling module 42 is configured to compile the test case by using a compiler to be tested to obtain a target compiling instruction;

a determining module 43, configured to determine whether the compiler has an exception based on a comparison result between the reference compiling instruction corresponding to the test case and the target compiling instruction. For the specific steps of the method for the device to execute the compiler test, reference may be made to the description in the foregoing method embodiment, which is not described herein again.

Correspondingly, an embodiment of the present disclosure further provides a device for storing test instructions, where the device includes:

the device comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for acquiring data related to a test instruction to be stored, and the data related to the test instruction comprises a plurality of pieces of feature data corresponding to each of a plurality of components of the test instruction and reference compiling data corresponding to each piece of feature data; the plurality of components comprise an instruction type, a control domain and an instruction address, wherein the control domain represents a control operation included by the test instruction;

the storage module is used for storing the data related to the test instruction through a tree structure, wherein a root node of any tree structure is used for storing a piece of target feature data in the instruction type and reference compilation data corresponding to the target feature data, and child nodes of the tree structure to leaf nodes of the tree structure are used for sequentially storing first associated feature data related to the target feature data in the control domain, reference compilation data corresponding to the first associated feature data, and second associated feature data related to the target feature data and the first associated feature data in the instruction address.

For the specific steps of the storage method for the device to execute the test instruction, reference may be made to the description in the above method embodiment, which is not described herein again.

Accordingly, an embodiment of the present disclosure further provides a device for generating a test case, where the test case is used to test a compiler, and a test instruction for constructing the test case is stored by the storage method described in the foregoing embodiment, and the device includes:

the obtaining module is used for obtaining feature data stored by each node in any path from a root node to a leaf node in the tree structure and reference compiling data corresponding to the feature data;

the splicing module is used for splicing the characteristic data stored in each node to obtain a test instruction, and splicing the reference compiling data corresponding to each characteristic data to obtain a reference compiling instruction corresponding to the test instruction;

and the generating module is used for generating a test case based on the obtained test instruction and generating a reference compiling instruction corresponding to the test case based on the reference compiling instruction corresponding to the test instruction.

The specific steps of the method for generating the test case executed by the apparatus may refer to the description in the above method embodiment, and are not described herein again.

Further, an embodiment of the present disclosure also provides an electronic device, as shown in fig. 5, where the electronic device includes a processor 51, a memory 52, and computer instructions stored in the memory 52 and executable by the processor 51, and when the processor 51 executes the computer instructions, the method in any of the foregoing embodiments is implemented.

The embodiments of the present disclosure also provide a computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the method of any of the foregoing embodiments.

Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.

From the above description of the embodiments, it is clear to those skilled in the art that the embodiments of the present disclosure can be implemented by software plus necessary general hardware platform. Based on such understanding, the technical solutions of the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, or the like, and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute the methods according to the embodiments or some parts of the embodiments of the present disclosure.

The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. A typical implementation device is a computer, which may take the form of a personal computer, laptop computer, cellular telephone, camera phone, smart phone, personal digital assistant, media player, navigation device, email messaging device, game console, tablet computer, wearable device, or a combination of any of these devices.

The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the apparatus embodiment, since it is substantially similar to the method embodiment, it is relatively simple to describe, and reference may be made to some descriptions of the method embodiment for relevant points. The above-described apparatus embodiments are merely illustrative, and the modules described as separate components may or may not be physically separate, and the functions of the modules may be implemented in one or more software and/or hardware when implementing the embodiments of the present disclosure. And part or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of the embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.

The foregoing is merely a detailed description of the embodiments of the disclosure, and it should be noted that modifications and decorations can be made by those skilled in the art without departing from the principle of the embodiments of the disclosure, and these modifications and decorations should also be regarded as the scope of protection of the embodiments of the disclosure.

Claims

1. A method for testing a compiler, the method comprising:

2. The method according to claim 1, wherein the test cases comprise functional test cases and/or stability test cases, each of the functional test cases comprises a single test instruction for testing the function of the compiler; and each stability test case is obtained by splicing a plurality of test instructions and is used for testing the stability of the compiler.

3. The method of claim 2, wherein the stability test case is one or more of the following:

the number of test instructions included in the stability test case is greater than a preset number;

4. The method according to claim 2 or 3, wherein the test cases include a functional test case and a stability test case, and the determining whether the compiler has an exception based on a comparison result between a reference compilation instruction corresponding to the test case and the target compilation instruction comprises:

5. The method according to any one of claims 1 to 4, wherein the test instruction comprises a plurality of components, the data related to the test instruction comprises a plurality of pieces of feature data corresponding to any one of the components and reference compiled data corresponding to any one piece of feature data, and the feature data corresponding to any one of the components is stored in association with the feature data corresponding to the rest of the components except for the any one of the components.

6. The method of claim 5, wherein the component parts comprise one or more of: the test instruction comprises an instruction type, a control domain and an instruction address, wherein the control domain is used for representing a control operation included by the test instruction.

7. The method according to claim 5 or 6, wherein the test case comprises a plurality of functional test cases for testing the compiler function, the test case is constructed based on data related to the test instruction stored in an instruction database, and a reference compiling instruction corresponding to the test case comprises:

8. The method according to claim 5 or 6, wherein the test cases include stability test cases for testing stability of the compiler, the test cases are constructed based on data related to test instructions stored in an instruction database, and reference compiling instructions corresponding to the test cases include:

for any item of tagged feature data in any component, determining associated feature data stored in association with the target feature data from the rest of the components except for any component in the plurality of components respectively;

9. The method according to any one of claims 6 to 8, wherein the data related to the test instruction is stored by a tree structure, a root node of any tree structure is used for storing a piece of target characteristic data in the instruction type and reference compiled data corresponding to the target characteristic data, and the child nodes to the leaf nodes of the tree structure are used for sequentially storing first associated characteristic data associated with the target characteristic data in the control domain, reference compiled data corresponding to the first associated characteristic data, and second associated characteristic data associated with the target characteristic data and the first associated characteristic data in the instruction address.

10. The method according to claim 9, wherein the constructing a test case based on data related to the test instruction stored in the instruction database, and the reference compiling instruction corresponding to the test case comprises:

11. The method according to any one of claims 1 to 10, wherein after determining that there is an exception in the compiler based on the comparison result between the reference compiled instruction and the target compiled instruction corresponding to the test case, the method further comprises:

12. A memory structure for testing instructions, said memory structure comprising: a tree structure storage structure;

13. A method for generating a test case, wherein the test case is used for testing a compiler, and test instructions for constructing the test case are stored in the storage structure according to claim 12, the method comprising:

14. An apparatus for testing a compiler, the apparatus comprising:

the building module is used for building a test case based on data related to the test instruction and stored in the instruction database and a reference compiling instruction corresponding to the test case;

the compiling module is used for compiling the test case by utilizing a compiler to be tested to obtain a target compiling instruction;

and the judging module is used for judging whether the compiler has abnormity or not based on the comparison result of the reference compiling instruction corresponding to the test case and the target compiling instruction.

15. An electronic device comprising a processor, a memory, and computer instructions stored in the memory for execution by the processor, the computer instructions when executed by the processor implementing the method of any of claims 1-11 or 13.

16. A computer-readable storage medium having computer instructions stored thereon that, when executed, implement the method of any one of claims 1-11 or 13.