CN116244703A

CN116244703A - Method and apparatus for fuzz testing

Info

Publication number: CN116244703A
Application number: CN202310174385.2A
Authority: CN
Inventors: 张雅琨
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2023-02-24
Filing date: 2023-02-24
Publication date: 2023-06-09

Abstract

The disclosure provides a fuzzy test method and device, relates to the field of artificial intelligence, and particularly relates to the field of deep learning. The specific implementation scheme is as follows: based on a structured data storage format Protobuf, carrying out structured processing on an application program interface API of the deep learning framework to be tested to obtain structured data, carrying out mutation processing on a preset test sample according to the structured data to obtain a mutation test sample, carrying out fuzzy testing on the API according to the mutation test sample to obtain a test result, and rapidly determining potential loopholes, safety problems and the like of codes of the deep learning framework, thereby improving the usability and safety of the deep learning framework.

Description

Fuzzy test method and device

Technical Field

The present disclosure relates to deep learning in artificial intelligence, and more particularly to a method and apparatus for fuzzy testing.

Background

With the development of artificial intelligence technology, deep learning is widely applied to various fields, and it is very important to select an appropriate deep learning framework before starting a deep learning project.

Among them, the safety of the deep learning frame is critical to the product of the deep learning frame, and the safety problem of the deep learning frame can lead to the heavy usability and safety of the product of the deep learning frame.

Disclosure of Invention

The present disclosure provides a method and apparatus for ambiguity testing that improves the effectiveness of the ambiguity testing.

According to a first aspect of the present disclosure, there is provided a method of ambiguity testing, comprising:

based on a structured data storage format Protobuf, carrying out structured processing on an application program interface API of the deep learning framework to be tested to obtain structured data;

performing mutation treatment on a preset test sample according to the structured data to obtain a mutation test sample;

and carrying out fuzzy test on the API according to the variant test sample, so as to obtain a test result.

According to a second aspect of the present disclosure, there is provided an apparatus for ambiguity testing, comprising:

the processing unit is used for carrying out structuring processing on an application program interface API of the deep learning framework to be tested based on a structured data storage format Protobuf to obtain structured data;

the mutation unit is used for carrying out mutation treatment on a preset test sample according to the structural data to obtain a mutation test sample;

and the testing unit is used for carrying out fuzzy test on the API according to the variation test sample to obtain a test result.

According to a third aspect of the present disclosure, there is provided an electronic device comprising:

At least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of the first aspect.

According to a fourth aspect of the present disclosure, there is provided a non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method according to the first aspect.

According to a fifth aspect of the present disclosure, there is provided a computer program product comprising: a computer program stored in a readable storage medium, from which it can be read by at least one processor of an electronic device, the at least one processor executing the computer program causing the electronic device to perform the method of the first aspect.

The method and the device for fuzzy test provided by the disclosure comprise the following steps: based on a structured data storage format Protobuf, carrying out structured processing on an application program interface API of a deep learning frame to be tested to obtain structured data, carrying out mutation processing on a preset test sample according to the structured data to obtain a mutation test sample, carrying out fuzzy testing on the API according to the mutation test sample to obtain a test result, carrying out the structured processing on the API based on the Protobuf to carry out mutation processing on the preset test sample so as to carry out the technical characteristics of the fuzzy testing on the API, and avoiding the complicated construction operation without constructing a model diagram in the embodiment, and avoiding the defect of low efficiency caused by the fuzzy testing of the whole dimension from the model diagram, so as to rapidly determine potential loopholes, safety problems and the like of codes of the deep learning frame, thereby improving the usability and safety of the deep learning frame.

It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.

Drawings

The drawings are for a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

FIG. 1 is a schematic diagram according to a first embodiment of the present disclosure;

FIG. 2 is a schematic diagram according to a second embodiment of the present disclosure;

FIG. 3 is a schematic diagram of a method of fuzzy testing according to an embodiment of the present disclosure;

FIG. 4 is a schematic diagram according to a third embodiment of the present disclosure;

FIG. 5 is a schematic diagram according to a fourth embodiment of the present disclosure;

FIG. 6 is a schematic diagram according to a fifth embodiment of the present disclosure;

fig. 7 is a block diagram of an electronic device for implementing a method of ambiguity testing in an embodiment of the present disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

For the convenience of the reader to understand the present disclosure, at least some of the terms of the present disclosure are now explained as follows:

artificial intelligence (Artificial Intelligence, AI) technology refers to technology that studies, develops theories, methods, techniques and application systems for simulating, extending and expanding human intelligence.

Fuzzing (Fuzzing) is a method of discovering software vulnerabilities by providing unexpected inputs to a target system and monitoring for anomalous results.

Application program interface (Application Programming Interface, API)), which refers to a collection of definitions, programs, and protocols through which computer software can communicate with each other. One of the main functions of the API is to provide a generic set of functions. The programmer can lighten the programming task by calling the API function to develop the application program. The API is also a middleware for providing data sharing for various platforms.

Byte refers to a basic unit of storage data.

Deep Learning (DL) is a sub-field in the Machine Learning (ML) field, and is an inherent rule and presentation hierarchy of Learning sample data, and information obtained in these Learning processes greatly helps interpretation of data such as text, images and sounds.

Before the deep learning project is started, it is important to select an appropriate deep learning framework, for example, the selection of an appropriate deep learning framework can perform a half-effort function. That is, different deep learning frameworks may be selected for different deep learning projects.

With the development of artificial intelligence technology, deep learning is widely applied to various fields such as image processing field, word processing field, and sound processing field, and so on. Accordingly, the deep learning framework has various products, such as an image processing model applied to the image processing field, a text recognition model applied to the word processing field, a dialogue model applied to the sound processing field, and the like.

The safety of the code of the deep learning frame is crucial to the product of the deep learning frame, and the safety problem of the code of the deep learning frame can lead to heavy frustration on the usability and safety of the product of the deep learning frame, and the current more effective floor drain hole mining method for the code of the deep learning frame is still in the exploring stage because of the complexity of the deep learning frame.

In some embodiments, the deep learning framework includes a plurality of APIs, and call relationships between APIs of the deep learning framework may be obtained, so as to generate a model graph (i.e., the model graph refers to a model of call relationships between APIs of the deep learning framework) based on the call relationships, so as to perform fuzzy test on the model graph as a whole, such as making a variation on the call relationships, so as to implement fuzzy test on the model graph, such as making a variation on the call relationships, and performing coverage detection, so as to implement fuzzy test on the model graph.

However, in one aspect, most of the parameters of the API are tensors (tensors), the number of dimensions of the Tensor, and the size of each dimension are variable. On the other hand, there are constraints on parameters of the APIs in each API, and not handling these constraints in advance makes it difficult to promote the ambiguity test depth, and a large number of input samples cannot touch deeper code logic.

To avoid at least one of the above problems, the present disclosure provides an inventive working technical idea: based on a structured data storage format Protobuf, carrying out structured processing on each API of the deep learning framework to obtain structured data, carrying out mutation processing on a preset test sample according to the structured data of any API to obtain a mutated test sample, and carrying out fuzzy testing on the API by combining the mutated test sample to finish the fuzzy testing on each API.

Based on the technical conception, the present disclosure provides a fuzzy test method and device, which are applied to deep learning in artificial intelligence to improve test effectiveness and reliability.

FIG. 1 is a schematic diagram of a first embodiment of the present disclosure, as shown in FIG. 1, a method of ambiguity testing of an embodiment of the present disclosure includes:

S101: based on a structured data storage format Protobuf, carrying out structured processing on an application program interface API of the deep learning framework to be tested to obtain structured data.

The execution body of the embodiment is an apparatus for fuzzy test (hereinafter simply referred to as a test apparatus), and the test apparatus may be a server, a terminal device, a processor, a chip, or the like, which is not listed here.

If the test device is a server, the server may be an independent server, or may be a server cluster, and the server may be a cloud server, or may be a local server, which is not limited in this embodiment.

In contrast, protobuf is a lightweight and efficient structured data storage format, is platform-independent, language-independent, and extensible, and can be used in the fields of communication protocols, data storage and the like.

In other words, on the one hand, protobuf can be applied to a platform, but is not necessarily related to the platform, and has a more flexible application scenario. On the other hand, protobuf can be free from language constraint, and has flexibility and universality. On the other hand, protobuf is expandable and thus has a strong adjustability.

Based on the technical concept of the disclosure, the disclosure is a fuzzy test performed from the dimension of each API, and because Protobuf has the characteristics, such as being independent of a platform and a language, light, efficient and adjustable, structured data of the API can be obtained based on Protobuf, thereby improving the effectiveness and reliability of subsequent fuzzy tests.

Illustratively, this step may be understood as: the testing device performs structural processing on each API of the deep learning framework and parameters (such as tensors) required by each API based on the characteristics of the structured data storage format of the Protobuf to obtain structured data.

S102: and carrying out mutation treatment on the preset test sample according to the structured data to obtain a mutation test sample.

The content and the number of the preset test samples are not limited in this embodiment, and may be determined based on the requirements, the history, the test, and the like.

The mutation process is understood to be that the preset test sample is adjusted so that the data content of the mutation test sample is different from that of the preset test sample. Accordingly, the method of the mutation processing is not limited in this embodiment, for example, the method of the mutation processing may be: and carrying out random adjustment processing on a preset test sample according to the structured data.

S103: and carrying out fuzzy test on the API according to the variant test sample to obtain a test result.

Illustratively, the present embodiment performs a mutation process by combining Protobuf to convert the fuzzy test for the overall model diagram in the above embodiment into a fuzzy test for the API.

Based on the above analysis, the present disclosure provides a method for fuzzy testing, the method comprising: based on a structured data storage format Protobuf, carrying out structured processing on an application program interface API of a deep learning frame to be tested to obtain structured data, carrying out mutation processing on a preset test sample according to the structured data to obtain a mutation test sample, and carrying out fuzzy testing on the API according to the mutation test sample to obtain a test result.

To facilitate the reader's understanding of the present disclosure, the implementation principles of the present disclosure will now be described in more detail in connection with fig. 2. Wherein, fig. 2 is a schematic diagram according to a second embodiment of the present disclosure, and as shown in fig. 2, a method for fuzzy testing according to an embodiment of the present disclosure includes:

s201: and acquiring a preset test sample.

The preset test sample is used for testing the code of the deep learning frame to determine whether the code of the deep learning frame has a bug.

It should be understood that, in order to avoid the cumbersome statement, the technical features of this embodiment that are the same as those of the above embodiment are not repeated.

For example, regarding the execution body of the present embodiment, reference may be made to the description of the above embodiments.

The number of APIs of the deep learning framework may be one or a plurality, and in general, the number of APIs of the deep learning framework is a plurality.

As can be seen from the technical concept of the present disclosure, in contrast to the fuzzy test performed on the overall model diagram in some embodiments, in this embodiment, the fuzzy test may be performed on each API, and if the number of APIs is multiple, the fuzzy test in this embodiment may be divided into two parts, where one part is the first fuzzy test and the other part is the non-first fuzzy test.

That is, when the test device obtains the preset test sample, the preset test sample corresponding to the first fuzzy test can be obtained; the test device can obtain the preset test sample corresponding to the non-first fuzzy test when the preset test sample is obtained or the non-first fuzzy test.

In combination with the above embodiment, the testing device needs to perform the mutation process before performing the fuzzy test on the API, and the mutation process may be the first mutation process or the non-first mutation process.

For example, if the test device obtains the preset test sample corresponding to the first fuzzy test when the preset test sample is the first fuzzy test, the mutation processing is the first mutation processing; if the test device obtains the preset test sample which is not the first fuzzy test, obtaining the preset test sample corresponding to the non-first fuzzy test, and changing the mutation processing into non-first mutation processing.

In some embodiments, if the mutation process is a first mutation process, the predetermined test sample is: based on the sample data loaded randomly and stored in bytes, a file (prototxt file) defining the network is generated.

For example, if the mutation process is a first mutation process, that is, when the preset test sample is obtained as a first fuzzy test, the preset test sample corresponding to the first fuzzy test is obtained, and then the preset test sample is a prototxt file obtained by random loading.

For example, data may be randomly loaded, where the data is stored in a byte manner, so that for convenience of distinction, we may refer to the data as sample data as shown in fig. 3, and convert the sample data into a prototxt file as shown in fig. 3, where the prototxt file is a preset test sample.

The prototxt file is a file based on Protobuf fall. With respect to the implementation of converting sample data into a prototxt file, the sample data may be converted into a prototxt file based on the characteristics of the structured data storage format of Protobuf.

In this embodiment, the sample data is converted into a prototxt file (i.e., a preset test sample), so that a fuzzy test for attaching a structural mutation based on protobuf is applied to a deep learning frame, thereby realizing rapid determination of potential loopholes and security problems of codes of the deep learning frame, and improving the usability and security of the deep learning frame.

In other embodiments, if the variance is not the first variance, the predetermined test sample is: and determining based on the historical variation test sample obtained by the historical variation processing.

For example, if the acquisition of the predetermined test sample is not the first acquisition of the predetermined test sample, for example, the N-th acquisition of the predetermined test sample (corresponding to the mutation process being the nth mutation process), the N-th acquisition of the predetermined test sample is determined based on the historical mutation test sample obtained from the previous N-1 mutation processes (i.e., the historical mutation process).

For example, if N is 10, the 10 th acquisition of the predetermined test sample is determined based on the historical mutation test sample obtained by the previous 9 mutation processes.

That is, the preset test sample obtained later is determined based on the historical mutation test sample obtained by the previous mutation process.

In contrast, in combination with the above embodiment, it is known that the mutation test sample is determined based on the structured data, and the structured data is obtained based on Protobuf, so that the mutation test sample is essentially based on prototxt files formed by Protobuf, so that the mutation test sample can be directly used without processing a text format of a preset test sample which is not acquired for the first time, thereby saving processing resources and improving the efficiency of the fuzzy test.

And, to the scene of first obtaining the preset test sample, and the scene of not first obtaining the preset test sample, obtain the preset test sample by different modes, can realize obtaining the flexibility and the variety of preset test sample.

In some embodiments, the predetermined test samples are: obtained from a stored test sample. Wherein, the test sample that keeps is: if the coverage rate of the fuzzy test based on the current mutation test sample is larger than the coverage rate of the fuzzy test based on the previous mutation test sample, the current mutation test sample is stored.

The historical variation test sample includes: a current variant test sample and a previous variant test sample.

For example, in combination with the above example, if the current mutation test sample is the 10 th mutation test sample, for convenience of distinction, we will refer to the coverage of the fuzzy test based on the 10 th mutation test sample as the first coverage. Accordingly, the previous mutation test sample is the 9 th mutation test sample, and for convenience of distinction, we will refer to the coverage of the fuzzy test based on the 9 th mutation test sample as the second coverage.

And if the first coverage rate is greater than the second coverage rate, saving the 10 th variant test sample, so as to obtain a saved test sample comprising the 10 th variant test sample.

It should be noted that the stored test sample may include the 10 th variant test sample, the 9 th variant test sample, and so on, and thus, the predetermined test sample may be at least a portion of the samples obtained from the stored test sample.

That is, in a scenario where the preset test sample is not acquired for the first time, the acquired preset test sample may be randomly acquired from among samples saved due to an increase in coverage.

In this embodiment, for a scenario in which a preset test sample is not acquired for the first time, the preset test sample is acquired by combining with the coverage rate, and the coverage rate of the fuzzy test is more comprehensive as the coverage rate is higher, so that the effectiveness and reliability of the fuzzy test can be improved.

S202: and carrying out structuring treatment on each API of the deep learning framework to be tested based on a structured data storage format Protobuf to obtain structured data.

The to-be-tested deep learning framework comprises a plurality of APIs, and when the to-be-tested deep learning framework is subjected to structural processing based on Protobuf, so that structural data is obtained.

Where, for an API, the API may be parameters of other APIs (such as call results of other APIs), the structuring process may be further understood as: and carrying out structuring processing on each API and parameters required by the API based on Protobuf to obtain structured data.

Wherein the structured data comprises: the name of the API, parameters of the API (including parameters of the API itself, as well as parameters of other APIs as needed), field attributes of the API, field types of the API, and so forth.

For example, for each API, the structured data includes the name of the API, parameters of the API, field attributes of the API, field types of the API, and so forth.

S203: and obtaining the structured data of any API from the structured data, and carrying out mutation processing on a preset test sample according to the obtained structured data to obtain a mutation test sample.

For example, as shown in fig. 3, a mutation process may be performed according to the structured data and a preset test sample to obtain a mutation test sample, and specifically, the structured data of one API may be randomly obtained from the structured data, so as to obtain the mutation test sample.

And the structured data of any API can be obtained from the structured data based on a preset obtaining strategy, so that a variation test sample is obtained. For example, the preset obtaining policy may be to obtain the structured data of any API from the structured data based on a preset sequence, so as to obtain the mutation test sample.

That is, the present embodiment does not limit the manner of obtaining the structured data of any API, so as to support flexibility, reliability, and validity of implementing the fuzzy test.

In connection with the above example, for each API, the structured data includes the name of the API, the parameters of the API, the field attributes of the API, the field types of the API.

Correspondingly, for any API, any API has the name of the API, if the name of any API is "abs", the testing device can perform mutation processing on the preset test sample according to the obtained structured data of the "abs" to obtain a mutation test sample.

In some embodiments, the obtained structured data comprises: the field attribute of any API and the field type of any API are used for representing whether any API comprises a target field or not; performing mutation processing on a preset test sample according to the obtained structured data to obtain a mutation test sample, wherein the mutation test sample comprises: if any API represented by the field attribute comprises a target field, performing mutation processing on a preset test sample according to the field type to obtain a mutation test sample.

Correspondingly, in combination with the above example, for any API, the "abs" has a field attribute and a field type, the field attribute of the "abs" indicates whether the "abs" includes a target field, and if the "abs" includes the target field, the mutation processing is performed on the preset test sample according to the field type of the "abs".

The target field may be determined based on a requirement, a history, a test, and the like, which is not limited in this embodiment.

Illustratively, the structured parameters of any API include parameters of the API, and the parameters may include parameters of the API itself, and may also include parameters of other APIs required by the API, that is, the number of parameters of the API may be plural, each parameter of the API may be a field, and before each parameter of the API, there may be a field attribute and a field type of the parameter.

In this embodiment, in a scenario where it is determined that any API includes a target field according to a field attribute, a mutation process is performed on a preset test sample according to a field type, so that the effectiveness and reliability of the mutation process can be achieved, and the accuracy and reliability of the obtained mutation test sample are further improved.

In some embodiments, the mutation processing is performed on the preset test sample according to the field type to obtain a mutated test sample, which includes the following steps:

a first step of: random data of a field type is generated.

Illustratively, if the field type is a floating point type, generating random data that is a floating point type; if the field type is an integer type, generating random data of the integer type.

And a second step of: and generating a variation test sample according to the random data and the preset test sample.

Correspondingly, after random data of a field type is randomly generated, mutation processing can be performed on a preset test sample based on the random data, so that a mutation test sample is obtained.

For example, the preset test sample may be subjected to an adjustment process based on random data, such as replacing corresponding data in the preset test sample with random data, and so on.

In this embodiment, by generating random data with a field type of any API, the random data relatively meets the requirement of the field type of any API, so that the generated variant test sample has stronger test performance, and the effectiveness and reliability of the fuzzy test are improved.

S204: and performing fuzzy test on any API according to the variant test sample to obtain a test result.

For example, as shown in fig. 3, the testing device may perform a fuzzy test on any API of the APIs according to the mutation test sample, so as to obtain a test result.

In this embodiment, for any API, structured data corresponding to the any API may be obtained from the structured data, so as to generate a mutation test sample corresponding to the any API in a targeted manner.

In some embodiments, S204 may include the steps of:

a first step of: and calling any API according to the mutation test sample.

Illustratively, the test device reads the variant test sample as an input parameter to call any API.

And a second step of: in response to invoking any API, the code of the deep learning framework crashes, and it is determined that the test result characterizes the code of the deep learning framework as having a vulnerability.

In the process that any API tested device calls based on the variant test sample, the code of the deep learning frame breaks down, so that the variant test sample can generate 'aggressiveness' to the code of the deep learning frame, and a test result representing that the code of the deep learning frame has loopholes can be generated.

In some embodiments, if the test device crashes the code of the deep learning frame during the process of calling any API based on the mutation test sample, as shown in fig. 3, the test device may save the mutation test sample, so as to modify the code of the deep learning frame in combination with the mutation test sample, to repair the bug of the code of the deep learning frame, and improve the security of the deep learning frame.

In this embodiment, any API is called by combining the variant test sample, so as to implement the fuzzy test from the dimension of the call of the single API, and compared with the integral test in the above embodiment, the efficiency of the fuzzy test can be improved, so that potential safety hazards of the code of the deep learning frame can be quickly found, and the availability and safety of the deep learning frame can be further improved.

As can be seen from the above description about acquiring the preset test sample, the preset test sample may be acquired in different manners for the first acquisition and the non-first acquisition, and if the preset test sample is not acquired for the first acquisition, the preset test sample may be acquired from the saved test sample, so in some embodiments, the test device may perform coverage instrumentation processing on the code of the deep learning frame, so that the coverage of the code of the deep learning frame may be referred to as the current coverage for the convenience of distinguishing when the acquired test device calls any API according to the variant test sample.

Correspondingly, the current coverage rate can be compared with the pre-stored previous coverage rate, and if the current coverage rate is increased relative to the previous coverage rate, the variation test sample can be used as a preset test sample of the next fuzzy test. For example, in the scenario of "coverage increase" as shown in fig. 3, the variant test sample may be taken as a preset test sample for the next fuzzy test.

Otherwise, if the current coverage is not increased relative to the previous coverage, the variant test sample may be discarded (discard), e.g., the variant test sample is not taken as a preset test sample for the next fuzzy test. For example, in a scenario where "coverage is not increased" as shown in fig. 3, the variant test sample may be discarded.

If the current coverage rate is the coverage rate of the first fuzzy test, the previous coverage rate is a preset value, for example, 0.

Fig. 4 is a schematic diagram of a third embodiment of the present disclosure, as shown in fig. 4, an apparatus 400 for ambiguity testing of an embodiment of the present disclosure, comprising:

the processing unit 401 is configured to perform a structuring process on an API of the deep learning framework to be tested based on a structured data storage format Protobuf, so as to obtain structured data.

And a mutation unit 402, configured to perform mutation processing on the preset test sample according to the structured data, so as to obtain a mutated test sample.

And the testing unit 403 is configured to perform fuzzy test on the API according to the variant test sample, so as to obtain a test result.

Fig. 5 is a schematic diagram of a fourth embodiment of the present disclosure, as shown in fig. 5, an apparatus 500 for ambiguity testing of an embodiment of the present disclosure, comprising:

the processing unit 501 is configured to perform a structuring process on an API of the deep learning framework to be tested based on a structured data storage format Protobuf, so as to obtain structured data.

The mutation unit 502 is configured to perform mutation processing on a preset test sample according to the structured data, so as to obtain a mutated test sample.

And the test unit 503 is used for performing fuzzy test on the API according to the variant test sample to obtain a test result.

In some embodiments, the number of APIs is multiple; as can be seen in fig. 5, the mutation unit 502 includes:

an obtaining subunit 5021, configured to obtain structured data of any API from the structured data; and the mutation subunit 5022 is configured to perform mutation processing on a preset test sample according to the obtained structured data to obtain a mutated test sample.

And the test unit 503 is configured to perform fuzzy test on any API according to the variant test sample, so as to obtain a test result.

In some embodiments, the obtained structured data comprises: the field attribute of any API and the field type of any API are used for representing whether any API comprises a target field or not; the mutation subunit 5022 is configured to, if the field attribute indicates that any API includes the target field, perform mutation processing on the preset test sample according to the field type, and obtain a mutated test sample.

In some embodiments, the variant subunit 5022 comprises:

and the first generation module is used for generating random data of a field type.

The second generation module is used for generating a variation test sample according to the random data and the preset test sample.

In some embodiments, as can be seen in conjunction with fig. 5, the test unit 503 includes:

the calling subunit 5031 is configured to call any API according to the mutation test sample.

A determining subunit 5032, configured to determine, in response to calling any API, that the code of the deep learning framework crashes, that the test result characterizes that the code of the deep learning framework has a bug.

In some embodiments, if the mutation process is a first mutation process, the predetermined test sample is: based on the randomly loaded and byte-wise stored sample data, a file is generated defining the network.

In some embodiments, if the mutation process is a non-first mutation process, the predetermined test sample is: and determining based on the historical variation test sample obtained by the historical variation processing.

In some embodiments, the predetermined test samples are: obtained from a stored test sample.

Wherein, the test sample that holds is: if the coverage rate of the fuzzy test based on the current mutation test sample is larger than the coverage rate of the fuzzy test based on the previous mutation test sample, the current mutation test sample is stored.

Fig. 6 is a schematic diagram according to a fifth embodiment of the present disclosure, as shown in fig. 6, an electronic device 600 in the present disclosure may include: a processor 601 and a memory 602.

A memory 602 for storing a program; the memory 602 may include a volatile memory (english: volatile memory), such as a random-access memory (RAM), such as a static random-access memory (SRAM), a double data rate synchronous dynamic random-access memory (DDR SDRAM), etc.; the memory may also include a non-volatile memory (English) such as a flash memory (English). The memory 602 is used to store computer programs (e.g., application programs, functional modules, etc. that implement the methods described above), computer instructions, etc., which may be stored in one or more of the memories 602 in a partitioned manner. And the above-described computer programs, computer instructions, data, etc. may be called upon by the processor 601.

The computer programs, computer instructions, etc., described above may be stored in one or more of the memories 602 in partitions. And the above-described computer programs, computer instructions, etc. may be invoked by the processor 601.

A processor 601 for executing a computer program stored in a memory 602 to implement the steps of the method according to the above embodiment.

Reference may be made in particular to the description of the embodiments of the method described above.

The processor 601 and the memory 602 may be separate structures or may be integrated structures integrated together. When the processor 601 and the memory 602 are separate structures, the memory 602 and the processor 601 may be coupled by a bus 603.

The electronic device in this embodiment may execute the technical scheme in the above method, and the specific implementation process and the technical principle are the same, which are not described herein again.

In the technical scheme of the disclosure, the related processes of collecting, storing, using, processing, transmitting, providing, disclosing and the like of the personal information of the user accord with the regulations of related laws and regulations, and the public order colloquial is not violated.

According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

According to an embodiment of the present disclosure, the present disclosure also provides a computer program product comprising: a computer program stored in a readable storage medium, from which at least one processor of an electronic device can read, the at least one processor executing the computer program causing the electronic device to perform the solution provided by any one of the embodiments described above.

Fig. 7 illustrates a schematic block diagram of an example electronic device 700 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile apparatuses, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing apparatuses. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 7, the apparatus 700 includes a computing unit 701 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 702 or a computer program loaded from a storage unit 708 into a Random Access Memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 may also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.

Various components in device 700 are connected to I/O interface 705, including: an input unit 706 such as a keyboard, a mouse, etc.; an output unit 707 such as various types of displays, speakers, and the like; a storage unit 708 such as a magnetic disk, an optical disk, or the like; and a communication unit 709 such as a network card, modem, wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.

The computing unit 701 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 701 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the respective methods and processes described above, for example, a method of blur testing. For example, in some embodiments, the method of ambiguity testing may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as the storage unit 708. In some embodiments, part or all of the computer program may be loaded and/or installed onto device 700 via ROM 702 and/or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the method of ambiguity testing described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the method of ambiguity testing in any other suitable manner (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.

The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service ("Virtual Private Server" or simply "VPS") are overcome. The server may also be a server of a distributed system or a server that incorporates a blockchain.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel or sequentially or in a different order, provided that the desired results of the technical solutions of the present disclosure are achieved, and are not limited herein.

The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure are intended to be included within the scope of the present disclosure.

Claims

1. A method of ambiguity testing, comprising:

2. The method of claim 1, wherein the number of APIs is a plurality; performing mutation processing on a preset test sample according to the structured data to obtain a mutation test sample, wherein the mutation test sample comprises:

obtaining structural data of any API from the structural data, and carrying out mutation treatment on the preset test sample according to the obtained structural data to obtain a mutation test sample;

and performing fuzzy test on the API according to the variant test sample to obtain a test result, wherein the fuzzy test comprises the following steps: and performing fuzzy test on any API according to the variant test sample to obtain the test result.

3. The method of claim 2, wherein the acquired structured data comprises: the field attribute of any API and the field type of any API are used for representing whether any API comprises a target field or not; performing mutation processing on the preset test sample according to the obtained structured data to obtain a mutation test sample, wherein the mutation test sample comprises the following components:

if the field attribute characterizes that any API comprises a target field, performing mutation processing on the preset test sample according to the field type to obtain the mutation test sample.

4. The method of claim 3, wherein mutating the preset test sample according to the field type to obtain the mutated test sample, comprising:

generating random data of the field type;

and generating the variation test sample according to the random data and the preset test sample.

5. The method of any one of claims 2-4, wherein performing a fuzzy test on the any API from the variant test sample to obtain the test result comprises:

calling any API according to the mutation test sample;

and responding to calling any API, determining that the code of the deep learning framework crashes, and determining that the test result characterizes the code of the deep learning framework to have loopholes.

6. The method of any one of claims 1-5, wherein if the mutation is a first mutation, the predetermined test sample is: based on the randomly loaded and byte-wise stored sample data, a file is generated defining the network.

7. The method of any one of claims 1-6, wherein if the mutation treatment is a non-first mutation treatment, the predetermined test sample is: and determining based on the historical variation test sample obtained by the historical variation processing.

8. The method of claim 7, wherein the predetermined test sample is: obtained from the stored test sample;

wherein, the test sample that holds is: if the coverage rate of the fuzzy test based on the current variation test sample is larger than that of the fuzzy test based on the previous variation test sample, the current variation test sample is stored;

the historical variation test sample comprises: the current variant test sample and the previous variant test sample.

9. An apparatus for ambiguity testing, comprising:

10. The apparatus of claim 9, wherein the number of APIs is a plurality; the mutation unit includes:

an obtaining subunit, configured to obtain structured data of any API from the structured data; the mutation subunit is used for carrying out mutation treatment on the preset test sample according to the obtained structured data to obtain a mutation test sample;

And the testing unit is used for carrying out fuzzy test on any API according to the variant test sample to obtain the test result.

11. The apparatus of claim 10, wherein the acquired structured data comprises: the field attribute of any API and the field type of any API are used for representing whether any API comprises a target field or not; and the mutation subunit is configured to, if the field attribute characterizes that the any API includes a target field, perform mutation processing on the preset test sample according to the field type, so as to obtain the mutation test sample.

12. The apparatus of claim 11, wherein the mutation subunit comprises:

the first generation module is used for generating random data of the field type;

and the second generation module is used for generating the variation test sample according to the random data and the preset test sample.

13. The apparatus of any of claims 10-12, wherein the test unit comprises:

a calling subunit, configured to call the any API according to the mutation test sample;

and the determining subunit is used for responding to the call of any API, determining that the code of the deep learning framework crashes, and the test result represents that the code of the deep learning framework has loopholes.

14. The apparatus of any one of claims 9-13, wherein if the mutation is a first mutation, the predetermined test sample is: based on the randomly loaded and byte-wise stored sample data, a file is generated defining the network.

15. The apparatus of any one of claims 9-14, wherein if the mutation process is a non-first mutation process, the predetermined test sample is: and determining based on the historical variation test sample obtained by the historical variation processing.

16. The apparatus of claim 15, wherein the predetermined test sample is: obtained from the stored test sample;

17. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.

18. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-8.

19. A computer program product comprising a computer program which, when executed by a processor, implements the steps of the method of any of claims 1-8.