CN116244162A

CN116244162A - Test method, test device, test equipment, test storage medium and test product

Info

Publication number: CN116244162A
Application number: CN202111483728.0A
Authority: CN
Inventors: 陈金龙
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2021-12-07
Filing date: 2021-12-07
Publication date: 2023-06-09

Abstract

The application discloses a testing method, a testing device, testing equipment, a storage medium and a testing product, and belongs to the technical field of computers. The method comprises the following steps: logic relation information corresponding to process events in at least two processes is obtained; generating clock information corresponding to the process event based on the logic relation information; determining a concurrent event set corresponding to the process event according to the clock information; carrying out event combination processing on the process events and the process events in the concurrent event set to obtain a concurrent event combination; and carrying out time sequence test on the concurrent event combination to obtain a test result. According to the technical scheme provided by the embodiment of the application, through the logic connection relation among the events in different processes, clock information capable of representing the sequence of the events is generated, so that the concurrent events in different processes are accurately identified and combined, and then the concurrent event combination is subjected to time sequence test to detect abnormal concurrent events in different processes, so that test results are obtained, automatic time sequence test is realized, and the accuracy of the time sequence test is improved.

Description

Test method, test device, test equipment, test storage medium and test product

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a testing method, apparatus, device, storage medium, and product.

Background

A distributed system is a system consisting of a set of computer nodes that communicate over a network to coordinate work in order to accomplish a common task. Distributed systems have emerged to perform computing, storage tasks with inexpensive, common machines that a single computer cannot perform. The purpose is to process more data with more machines.

In the related art, the work of performing time sequence test and concurrent event identification on a distributed system needs to rely on understanding and experience of a developer or a tester on a tested distributed algorithm to judge whether related events in the distributed system are concurrent or not and construct the sequence of execution of the related concurrent events.

In the related art, the time sequence test of the distributed system cannot be modeled, the identification rate of concurrent events is low, the completeness of the time sequence test is poor, and the degree of automation is low.

Disclosure of Invention

The embodiment of the application provides a testing method, a testing device, testing equipment, a storage medium and a testing product, which can improve the identification accuracy of concurrent events, realize the automatic modeling processing of time sequence testing on the whole, improve the automation degree of the time sequence testing, improve the accuracy and completeness of the time sequence testing and reduce the complexity of the time sequence testing.

According to one aspect of embodiments of the present application, there is provided a test method, the method comprising:

acquiring logic relation information corresponding to process events in at least two processes, wherein the logic relation information is used for representing logic connection relations between the process events;

generating clock information corresponding to the process event based on the logic relation information, wherein the clock information is used for representing the corresponding sequence relation of the process event in the at least two processes;

determining a concurrent event set corresponding to the process event according to the clock information;

carrying out event combination processing on the process event and the process event in the concurrent event set to obtain a concurrent event combination;

and carrying out time sequence test on the concurrent event combination to obtain a test result, wherein the time sequence test is used for detecting abnormal concurrent events, and the test result is used for representing the execution condition of the concurrent event combination.

According to an aspect of embodiments of the present application, there is provided a test apparatus, the apparatus comprising:

the event relation acquisition module is used for acquiring logic relation information corresponding to process events in at least two processes, wherein the logic relation information is used for representing logic connection relations between the process events;

The event clock generation module is used for generating clock information corresponding to the process event based on the logic relation information, and the clock information is used for representing the corresponding sequence relation of the process event in the at least two processes;

the concurrent event determining module is used for determining a concurrent event set corresponding to the process event according to the clock information;

the concurrent event combination module is used for carrying out event combination processing on the process event and the process event in the concurrent event set to obtain a concurrent event combination;

the event combination test module is used for carrying out time sequence test on the concurrent event combination to obtain a test result, wherein the time sequence test is used for detecting abnormal concurrent events, and the test result is used for representing the execution condition of the concurrent event combination.

According to an aspect of the embodiments of the present application, there is provided a computer apparatus including a processor and a memory, where at least one instruction, at least one program, a code set, or an instruction set is stored, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the above-mentioned test method.

According to one aspect of embodiments of the present application, there is provided a computer-readable storage medium having stored therein at least one instruction, at least one program, a set of codes, or a set of instructions, the at least one instruction, the at least one program, the set of codes, or the set of instructions being loaded and executed by a processor to implement the above-described test method.

According to one aspect of embodiments of the present application, there is provided a computer program product comprising computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from a computer-readable storage medium, the processor executing the computer instructions, causing the computer device to execute to implement the test method described above.

The technical scheme provided by the embodiment of the application can bring the following beneficial effects:

by acquiring the logic connection relation between the events in different processes, clock information of the events in different processes can be generated to determine the sequence relation between the events in different processes, so that concurrent events in different processes can be automatically and accurately identified according to the clock information of the events, manual judgment is not needed, and the identification accuracy of the concurrent events is improved; after the concurrent events are identified, the identified concurrent events can be combined, and the time sequence test is carried out on the concurrent event combination so as to detect abnormal concurrent events in different processes, so that test results which can represent the execution condition of the concurrent event combination in the time sequence test are obtained, the automatic modeling processing of the time sequence test is realized on the whole, the automation degree of the time sequence test is improved, the accuracy and the completeness of the time sequence test are improved, and the complexity of the time sequence test is reduced.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic diagram of an alternative architecture of a distributed system for use in a blockchain system in accordance with embodiments of the present invention;

FIG. 2 is a schematic diagram of an alternative block structure according to an embodiment of the present invention;

FIG. 3 is a flow chart diagram of a test method provided in one embodiment of the present application;

FIG. 4 is a second flowchart of a test method provided in one embodiment of the present application;

FIG. 5 illustrates a schematic diagram of a timing diagram;

FIG. 6 illustrates a schematic diagram of a vector clock diagram;

FIG. 7 illustrates a schematic diagram of an event vector clock graph;

FIG. 8 is a flow chart III of a test method provided by one embodiment of the present application;

FIG. 9 illustrates a schematic diagram of a model module in a distributed system timing test tool;

FIG. 10 is a schematic diagram illustrating the structure of an algorithm module in a distributed system timing test tool;

FIG. 11 illustrates a schematic diagram of a test results presentation page;

FIG. 12 is a schematic diagram illustrating a distributed system timing test platform execution flow;

FIG. 13 illustrates a block diagram of a distributed system timing test solution;

FIG. 14 is a block diagram of a test apparatus provided in one embodiment of the present application;

fig. 15 is a block diagram of a computer device according to an embodiment of the present application.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

The system according to the embodiment of the present invention may be a distributed system formed by connecting a client and a plurality of nodes (any form of computing device in an access network, such as a server and a user terminal) through a network communication.

A distributed system is a loosely coupled system of multiple processors interconnected by communication lines. The remaining processors and corresponding resources are remote from the point of view of a processor in the system, and only their own resources are local. The method has the characteristics of distribution, autonomy, parallelism, global property and the like.

The user terminal comprises, but is not limited to, mobile phones, computers, intelligent voice interaction devices, intelligent home appliances, vehicle-mounted terminals, game hosts, electronic book readers, multimedia playing devices, wearable devices and other electronic devices. A client in which an application can be installed.

In the embodiment of the present application, the application programs include, but are not limited to, a finance type application program, a news type application program, a social type application program, an interactive entertainment type application program, a browser application program, a shopping type application program, a content sharing type application program, a Virtual Reality (VR) type application program, an augmented Reality (Augmented Reality, AR) type application program, and the like, which are not limited thereto.

The server is used for providing background service for the client of the application program in the user terminal. For example, the server may be a background server of the application program described above. The server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs (Content Delivery Network, content delivery networks), basic cloud computing services such as big data and artificial intelligent platforms, and the like. Optionally, the server provides background services for applications in multiple user terminals simultaneously.

Alternatively, the user terminal and the server may be directly or indirectly connected through wired or wireless communication, which is not limited herein.

Taking a distributed system as an example of a blockchain system, referring To fig. 1, fig. 1 is a schematic diagram of an alternative architecture of a distributed system 100 applied To a blockchain system according To an embodiment of the present invention, where the architecture is formed by a plurality of nodes (arbitrary computing devices in an access network, such as servers and user terminals) and clients, and a Peer-To-Peer (P2P, peer To Peer) network is formed between the nodes, where the P2P protocol is an application layer protocol running on top of a transmission control protocol (TCP, transmission Control Protocol) protocol. In a distributed system, any machine, such as a server, a terminal, may join to become a node, including a hardware layer, an intermediate layer, an operating system layer, and an application layer.

Referring to the functionality of each node in the blockchain system shown in fig. 1, the functions involved include:

1) The routing, the node has basic functions for supporting communication between nodes.

Besides the routing function, the node can also have the following functions:

2) The application is used for being deployed in a block chain to realize specific service according to actual service requirements, recording data related to the realization function to form recorded data, carrying a digital signature in the recorded data to represent the source of task data, sending the recorded data to other nodes in the block chain system, and adding the recorded data into a temporary block when the source and the integrity of the recorded data are verified by the other nodes.

For example, the services implemented by the application include:

2.1 Wallet for providing electronic money transactions, including initiating a transaction (i.e., sending a transaction record of the current transaction to other nodes in the blockchain system, the other nodes, after verification, storing record data of the transaction in a temporary block of the blockchain in response to acknowledging that the transaction is valid; of course, the wallet also supports inquiry of remaining electronic money in the electronic money address;

2.2 The shared account book is used for providing the functions of storing, inquiring, modifying and the like of account data, sending record data of the operation on the account data to other nodes in the blockchain system, and after the other nodes verify to be effective, storing the record data into a temporary block as a response for acknowledging that the account data is effective, and also sending confirmation to the node initiating the operation.

2.3 A computerized agreement that can execute the terms of a contract, implemented by code deployed on a shared ledger for execution when certain conditions are met, for completing automated transactions based on actual business demand codes, such as querying the physical distribution status of the goods purchased by the buyer, transferring the electronic money of the buyer to the merchant's address after the buyer signs for the goods; of course, the smart contract is not limited to executing the contract for the transaction, and may execute a contract that processes the received information.

3) The blockchain comprises a series of blocks (blocks) which are connected with each other according to the generated sequence time, the new blocks are not removed once being added into the blockchain, and record data submitted by nodes in the blockchain system are recorded in the blocks.

Referring to fig. 2, fig. 2 is an optional Block Structure (Block Structure) provided in an embodiment of the present invention, where each Block includes a hash value of a transaction record stored in the Block (hash value of the Block) and a hash value of a previous Block, and each Block is connected by the hash value to form a Block chain. In addition, the block may include information such as a time stamp at the time of block generation. The Blockchain (Blockchain), which is essentially a de-centralized database, is a string of data blocks that are generated in association using cryptographic methods, each of which contains associated information that is used to verify the validity (anti-counterfeiting) of its information and to generate the next block.

The distributed system has asynchronous and concurrent characteristics naturally, is a challenge for identifying the concurrent events of the distributed system, traversing the sequential combination of all the concurrent events and implementing the sequential execution of the concurrent events, and in the related technology, the identification of the concurrent events of the distributed system needs to depend on the understanding of development or testers on the tested distributed algorithm, and experience, has no unified test modeling tool, and has low concurrent event identification rate and poor completeness of time sequence test. Therefore, the embodiment of the application provides a testing method, which can be summarized as a distributed system time sequence testing method based on vector clock analysis, which is used for modeling a tested distributed algorithm in advance, constructing time sequence diagrams of different events of different nodes, calculating the vector clock of each event, analyzing and identifying concurrent events in the distributed system, and finally generating sequential execution sequence combinations of all the concurrent events for performing time sequence testing so as to solve the problems.

Before describing the method embodiments provided herein, related terms or nouns that may be involved in the method embodiments of the present application are briefly described, so as to be understood by those skilled in the art of the present application.

A logic clock (clock) is a concept proposed to distinguish physical clocks in reality, and is a time mechanism for distinguishing an occurrence sequence of events in a distributed system.

The vector clock is an algorithm for describing the causal relation of the events in the distributed system, the order relation of any two events can be obtained based on the vector clock, and the result is either causal (sequential) or causal (simultaneous) free.

Software development and promotion flow management platform: managing software version development, test lifting, defect submission, regression testing and publishing the whole life cycle.

Referring to fig. 3, a flowchart of a testing method according to an embodiment of the present application is shown. The method can be applied to computer equipment, wherein the computer equipment is electronic equipment with data computing and processing capabilities. The method may include the following steps (310-350).

Step 310, obtaining logic relationship information corresponding to process events in at least two processes.

The at least two processes refer to at least two processes in the distributed system, and the at least two processes independently run. In a distributed system, a server or a process running on a server is often referred to as a node, where the nodes are interconnected by a network. Alternatively, the at least two processes may correspond to different physical devices, or may be different processes independently running in the same device.

The process event is an event executed in a process. The event category of the process event comprises an in-process event, a received message event and a sent message event.

The logical relationship information is used to characterize the logical connection relationship between process events. In the distributed system, the events have logic connection relation, and for the events in the same process, the events in the same process correspond to different execution sequences in the local process, and have sequence relation. For events within different processes, there may be a messaging process between different processes, where a send message event occurs in one process, then a receive message event may occur on the process receiving the message.

In an exemplary embodiment, the above logical relationship information may be characterized by a timing diagram. Accordingly, as shown in fig. 4, the implementation process of the step 310 includes the following steps (311 to 315), and fig. 4 shows a second flowchart of the test method provided in one embodiment of the present application.

Step 311, determining a time axis corresponding to each of the at least two processes.

Each process in the distributed system corresponds to a time axis for marking the order of execution of events on the corresponding process. The time axis may be a physical time axis or a logical time axis.

Alternatively, in the above-mentioned timing chart, each process corresponds to a time axis, and the time axes of the respective processes are distributed in parallel.

In one possible implementation manner, a distributed system timing test tool can be constructed based on the technical scheme provided by the embodiment of the application. The time sequence testing tool comprises a time axis constructor, wherein the time axis constructor has the functions of generating a time axis, adding events to the time axis, adding interactions to the time axis, acquiring the number of events of the time axis and the like. In step 311, a time axis corresponding to each of the at least two processes is generated based on the time axis builder.

Step 312, determining event nodes corresponding to the process events on the time axis.

Each process event corresponds to an event node on a time axis of a process where the event is located, and the event node can represent the process event in the time sequence diagram.

In an exemplary embodiment, event information corresponding to a process event is acquired; and adding event nodes corresponding to the process events on the corresponding time axes according to the event information of the process events.

Optionally, the distributed system timing testing tool further comprises an event builder. Based on the event builder, event information of the process event can be obtained, wherein the event information comprises, but is not limited to, event identification, event description information and event attribute information. Therefore, the event builder has functions of acquiring event identification, acquiring event description information, acquiring event attribute information, and the like.

Correspondingly, after the time axis constructor constructs the time axis corresponding to each process, event nodes corresponding to process events in the process are added on the time axis through event information acquired by the event constructor. Optionally, determining an execution order of the process events on the process where the events are located based on the event information; and determining the time axis position information of the event node corresponding to the process event on the time axis according to the execution sequence of the process event. For example, if the process event 01 is the first process event in the process 0, the event node corresponding to the process event 01 is the first node on the time axis corresponding to the process 0. Optionally, the above-described execution order corresponds to a time axis position order of the nodes.

Step 313, determining the interaction event corresponding to the process event and the message transmission direction between the process event and the interaction event when the event type of the process event is the sending message event or the receiving message event.

The event attribute information in the event information is used for representing the event type of the process event. Alternatively, the event type of the process event may be determined according to the event attribute information described above. The event types include a send message event and a receive message event.

The interactive event refers to a process event having an interactive relationship with a process event in other processes, where the process where the interactive event is located is different from the process where the process event is located, and the other processes refer to processes except for a local process where the process event is located in the at least two processes. If the process event is a sending message event or a receiving message event, the process event has a corresponding interaction event, and the interaction event corresponding to the process event can be determined according to the transmission message corresponding to the process event.

The message passing direction is from the interaction starting event to the interaction ending event. The message transmission direction between the process event and the interaction event can be determined according to the event type of the process event.

If the process event is a message sending event, it can be determined that the process event is an interaction starting event, and the corresponding interaction event is an interaction ending event. In this case, the message passing direction is transmitted from the process event to the interaction event.

If the process event is a received message event, it may be determined that the process event is an interaction end event, and the corresponding interaction event is an interaction start event. In this case, the message passing direction is transmitted from the interactive event to the process event.

In the above embodiment, the distributed system timing testing tool further includes an interactivity event builder, where the interactivity event builder may obtain interactivity event information, where the interactivity event information includes an interactivity identifier, interactivity event description information, interactivity start event information, and interactivity end event information. Optionally, under the condition that the event type of the process event is a sending message event or a receiving message event, acquiring interaction event information corresponding to the process event, and determining the interaction event corresponding to the process event according to the interaction event information. Since the interactivity event information includes interactivity start event information and interactivity end event information, the message transfer direction may be determined according to the interactivity event information.

Step 314, based on the message passing direction, constructing a vector edge between the event node corresponding to the process event and the event node corresponding to the interaction event.

The vector edge refers to an edge having a connection direction. The direction of the quality edge is consistent with the message passing direction.

If the message transmission direction is transmitted from the process event to the interaction event, the vector edge is constructed by taking an event node corresponding to the process event as a starting point and taking an event node corresponding to the interaction event as an ending point.

If the message transmission direction is transmitted from the interactive event to the process event, the vector edge is constructed by taking an event node corresponding to the interactive event as a starting point and taking an event node corresponding to the process event as an ending point.

In the above embodiment, the event builder may obtain the event attribute information, so as to determine an event type of the process event. Under the condition that the event type of the process event is a sending message event or a receiving message event, acquiring interaction event information corresponding to the process event through the interaction event constructor so as to determine the interaction event and a message transmission direction; then, the adding interactive function of the time axis constructor may add a vector edge consistent with the message passing direction between the progress event and the interactive event.

Step 315, obtaining a timing diagram corresponding to the process event based on the time axis, the event node and the vector edge.

The timing diagram is used to characterize the logical relationship information.

And establishing a time axis corresponding to each process, adding event nodes corresponding to process events in the respective process on each time axis, and adding vector edges between the event nodes with interaction relation to obtain a time sequence diagram corresponding to the distributed system. Through the time sequence diagram, the logical connection relation among the events in each process in the distributed system can be intuitively seen, and modeling of a plurality of events in each process of the distributed system is realized.

In one example, as shown in fig. 5, a schematic diagram of a timing diagram is shown. The timing chart shown in fig. 5 includes a time axis 50 corresponding to the process 0 (P0), a time axis 51 corresponding to the process 1 (P1), a time axis 52 corresponding to the process 2 (P2), and a vector edge 54 between the event node 53 and the event node 53 on the time axis. The node pattern of the event node 53 characterizes event types, which are shown in fig. 5 including in-process events, send message events, and receive message events.

Step 320, generating clock information corresponding to the process event based on the logic relationship information.

The clock information is used for representing the corresponding sequence relation of the process events in at least two processes. Optionally, the clock information includes timestamp data corresponding to the process events on the at least two processes, and according to the timestamp data of each process event on each process, an execution sequence relationship between each process event can be determined.

The clock information includes, but is not limited to, physical clocks, logical clocks, vector clocks, mixed clocks, etc. corresponding to the events, which is not limited in the implementation of the present application.

In an exemplary embodiment, the above-described clock information may be characterized by a vector clock. Accordingly, as shown in fig. 4, the implementation process of the above step 320 includes the following steps (321 to 326).

Step 321, determining time axis position information corresponding to the event node based on the time chart.

The time axis position information is used to characterize the order of the position arrangement of the event nodes on the time axis.

In the above-mentioned time chart, the event nodes corresponding to the process events in each process are arranged on the time axis corresponding to each process, so that the position arrangement order of each event node on the time axis can be determined according to the above-mentioned time chart. For example, it is determined what node the event node is on the time axis.

Step 322, determining first timestamp data corresponding to the event node on the first process according to the time axis position information.

The first process refers to a process corresponding to a time axis where the event node is located in at least two processes.

According to the position arrangement order of the event nodes on the time axis, the first timestamp data corresponding to the event nodes on the local process can be determined. Optionally, the first timestamp data is determined according to the position arrangement order based on a preset mapping relationship. For example, the sequence of the positions of the event nodes on the time axis is n, so that the corresponding timestamp data of the event nodes on the local process is also n, and n is a positive integer.

Step 323, determining a corresponding preamble node of the event node in the timing diagram.

The preamble node includes an event node whose position is arranged in order before the event node in the local process. When the event node is connected with a vector edge pointing to the event node, the event node at the start point of the vector edge is also a corresponding preamble node of the event node in the time sequence diagram.

Step 324, obtain the corresponding time stamp data of the preamble node on the second process.

The second process herein does not mean a certain process, and the second process means another process other than the first process among at least two processes. Thus, the number of the second processes may be one or two or more.

The timestamp data corresponding to the preamble node on the second process includes timestamp data corresponding to the preamble node on each second process.

Step 325, determining second timestamp data corresponding to the event node on the second process according to the timestamp data.

Optionally, the corresponding time stamp data of the preamble node on each second process are respectively overlapped according to the corresponding process, so as to obtain the corresponding second time stamp data of the event node on the second process. The second event stamp data includes corresponding time stamp data of the event node on each second process. The corresponding time stamp data of the event node on each second process is accumulated data of the corresponding time stamp data of each preamble node on each second process.

In the case that the event node has no preamble node, the second timestamp data may be 0, that is, the timestamp data of the event node on other processes is marked as 0. The event node is the first event node when the event node does not have a preamble node, and the process event corresponding to the event node is an in-process event.

In step 326, a vector clock corresponding to the process event is generated based on the first timestamp data and the second timestamp data.

The vector clock is used to characterize the clock information.

In an exemplary embodiment, the first timestamp data is timestamp data corresponding to a process event on a first process, the second timestamp data is timestamp data corresponding to a process event on each second process, and the vector clock may be constructed based on timestamp data corresponding to the process event on the first process and timestamp data corresponding to each second process. The vector clock is a vector representing the execution sequence of the process event in at least two processes, and the vector dimension of the vector clock is consistent with the number of the processes of the at least two processes.

In one example, as shown in fig. 6, a schematic diagram of a vector clock diagram is illustratively shown. The vector clock diagram shown in fig. 6 is determined based on the timing diagram shown in fig. 5, and in comparison with the timing diagram shown in fig. 5, the vector clock 55 corresponding to the event node 53 is determined according to the time axis position of the event node 53 and the time stamp data of the preamble node in fig. 6, and the vector clock 55 is marked above the corresponding event node 53 in fig. 6.

In an exemplary embodiment, the distributed system timing test tool further includes an algorithm (Algorithms) module and a vector clock (generator) generation module. The algorithm module comprises the functions of vector clock generation, combination, comparison, sequencing, concurrent event identification and the like, and is a core module for concurrent event generation. The vector clock generation module is a module for converting the timing diagram into a vector clock, mainly realizes vector clock base class and basic function, and has the functions of generation_vclock (generating vector clock), get_axis_nums (obtaining the time axis number of the vector clock), get_event_nums (obtaining the event number of the vector clock) and the like at present. The model module already builds a time sequence diagram of the distributed system, and the vector clock generation module can call a vector clock generation algorithm contained in the algorithm module to convert the time sequence diagram into a vector clock.

Step 330, determining a concurrent event set corresponding to the process event according to the clock information.

And determining a concurrent event set corresponding to the process event according to the vector clock of the process event. The vector clock described above may characterize the overall sequence relationship between process events. By whole order, it is meant that any pair of elements within a collection are comparable to each other under a certain relationship. Therefore, by comparing vector clocks of different process events, whether the different process events are concurrent events or not can be judged, and further concurrent events corresponding to each process event can be determined to form the concurrent event set.

In an exemplary embodiment, the concurrency event may be determined by a vector clock. Accordingly, as shown in fig. 4, the implementation process of the above step 330 includes the following steps (331-333).

And step 331, comparing the target vector clock corresponding to the target process event with vector clocks corresponding to other process events to obtain a vector clock comparison result.

The target process event is any process event, and the other process events are process events except the target process event in the process events.

The vector clock comparison result comprises a vector clock size comparison result of the target process event and other process events. The size comparison result comprises a first comparison result, a second comparison result and a third comparison result, wherein the first comparison result is that a target vector clock corresponding to a target process event is larger than vector clocks corresponding to other process events; the second comparison result is that the target vector clock corresponding to the target process event is smaller than the vector clocks corresponding to other process events; the third comparison result indicates that the size relationship does not exist between the target vector clock corresponding to the target process event and the vector clocks corresponding to other process events.

The above-mentioned relation is that the time stamp data of each dimension in the pointing quantity clock is larger than the time stamp data of the corresponding dimension in another vector, the above-mentioned relation is that the time stamp data of each dimension in the pointing quantity clock is smaller than the time stamp data of the corresponding dimension in another vector, and for other cases, there is no size relation.

And step 332, determining a target concurrency event corresponding to the target process event according to the vector clock comparison result.

The vector clock corresponding to the target concurrent event accords with the target vector clock and the concurrent event clock condition.

And determining the process event corresponding to the third comparison result as a target concurrent event corresponding to the target process event. That is, it is stated that there is no size relationship between the vector clock corresponding to the target process event and the vector clock corresponding to the target concurrent event.

If the size relation does not exist between the target vector clock corresponding to the target process event and the vector clock corresponding to a certain process event, the process event can be determined to be the target concurrency event corresponding to the target process event.

Step 333, obtaining a target concurrency event set corresponding to the target process event based on the target concurrency event.

The target concurrent event set is an event set composed of the target concurrent events. The target concurrent event set is a concurrent event set corresponding to a target process event.

For each process event, the same operations as those of the target process event in steps 331 to 333 may be performed, and the concurrent event corresponding to each process event may be determined, and further, the concurrent event set corresponding to each process event may be determined.

In one example, as shown in FIG. 7, a schematic diagram of an event vector clock graph is illustratively shown. For ease of description, each event node 53 in fig. 6 is labeled as event node A, B, C, D, E, F, G, H, I, J, K in fig. 7. Accordingly, the vector clocks of event node A are [1, 0], event node B are [2,0,0], event node C are [3,0,2], event node D are [0,1,0], event node E are [0,2,0], event node F are [2,3,0], event node G are [2,4,0], event node H are [0, 1], event node I are [0, 2], event node J are [1,0,3], event node K are [2, 4].

Based on the vector clocks of the event nodes in fig. 7, the concurrent events in fig. 7 can be identified. The partial concurrency event recognition results are as follows:

the event node concurrent with the event node a includes: event node D, event node E, event node H, event node I, event node J;

the event node concurrent with the event node B includes: event node D, event node E, event node H, event node I, event node J;

the event node concurrent with the event node C includes: event node D, event node E, event node F, event node G;

the event node concurrent with the event node D includes: event node A, event node B, event node C, event node H, event node I, event node J; … ….

And 340, carrying out event combination processing on the process events and the process events in the concurrent event set to obtain a concurrent event combination.

For any process event, grouping the process event and the concurrent event corresponding to the process event to obtain a process event group, wherein the process event group comprises the process event and at least one concurrent event corresponding to the process event, and the process events in the process event group are arranged to obtain a concurrent event combination of each execution sequence corresponding to the process event group.

And carrying out the operation on each process event, so that the concurrent event combination of all execution sequences corresponding to all the concurrent events can be obtained.

In an exemplary embodiment, as shown in FIG. 4, the implementation of step 340 described above includes the following steps (341-342).

Step 341, combining the process event with the process event in the concurrent event set to obtain a concurrent event pair.

And respectively combining the process events with each process event in the concurrent event set to obtain a concurrent event pair.

In one possible implementation manner, the concurrent event pair is a concurrent event pair formed by process events that are mutually concurrent events in at least two processes. Optionally, the concurrent event pairs are unordered concurrent event pairs, i.e. there is no permutation relation between the concurrent events in the concurrent event pairs. For each process event, the concurrent event pairs corresponding to the process event can be obtained, but because the concurrent events are relative relationships and the concurrent event pairs are unordered concurrent event pairs, some concurrent event pairs can be repeatedly obtained in the process of traversing the process of generating the concurrent event pairs, and the repeated concurrent event pairs are deleted to obtain the concurrent event pairs formed by the process events which are mutually concurrent events in at least two processes.

In another possible implementation manner, the above concurrent event pairs are ordered concurrent event pairs, the process event is preceding, and the process event in the concurrent event set is following. Because the concurrent event pairs are ordered concurrent event pairs, repeated concurrent event pairs cannot be obtained in the process of generating the concurrent event pairs through traversal.

And 342, arranging the process events in the concurrent event pairs to obtain a concurrent event combination of at least one execution sequence corresponding to the concurrent event pairs.

In an exemplary embodiment, in the case that the pair of concurrent events is an ordered pair of concurrent events, the event arrangement is performed according to the arrangement order of two concurrent events in the ordered pair of concurrent events, so that a combination of concurrent events consistent with the order of events in the ordered pair of concurrent events can be obtained. For the first process event and the second process event which are concurrent events, two pairs of ordered concurrent events are respectively { first process event, second process event }, { second process event, first process event }, and the corresponding concurrent events are combined into (first process event, second process event), (second process event, first process event). And for the first process event and the second process event which are concurrent events, the concurrent event combination of the two execution sequences is obtained.

And under the condition that the concurrent event pair is an unordered concurrent event pair, carrying out event arrangement on two concurrent events in the concurrent event pair to obtain a concurrent event combination of all execution sequences corresponding to the concurrent event pair.

In an exemplary embodiment, the concurrent event pair includes a first process event and a second process event, the first process event and the second process event being concurrent events with each other. Accordingly, as shown in fig. 8, the implementation process of step 342 includes the following steps (342 a-342 b), and fig. 8 shows a flowchart three of the test method provided in one embodiment of the present application.

In step 342a, the first process event is arranged before the second process event, so as to obtain a concurrent event combination of the concurrent events corresponding to the first execution sequence.

Optionally, the concurrent events of the first execution sequence are combined into (first process event, second process event).

Optionally, delay time setting processing is performed on the first process event and the second process event to obtain a first execution delay time corresponding to the first process event and a second execution delay time corresponding to the second process event. Wherein the first execution delay time is smaller than the second execution delay time. The arrangement sequence among the process events can be adjusted by setting the execution delay time.

In step 342b, the first process event is arranged after the second process event, so as to obtain a concurrent event combination of the concurrent event pair corresponding to the second execution sequence.

Optionally, the concurrent events of the second execution sequence are combined into (second process event, first process event).

Optionally, delay time setting processing is performed on the first process event and the second process event to obtain a first execution delay time corresponding to the first process event and a second execution delay time corresponding to the second process event. Wherein the first execution delay time is greater than the second execution delay time. The arrangement sequence among the process events can be adjusted by setting the execution delay time.

It should be noted that, the first process event and the second process event are any pair of concurrent events of the system, and after the step 342b is executed, all possible combinations of concurrent events may be obtained. Optionally, all possible concurrent event combinations include at least one execution order of the concurrent event combinations for each concurrent event pair.

In the example corresponding to fig. 7, the concurrent events in fig. 7 are combined, so as to obtain a concurrent event combination of all execution sequences corresponding to the concurrent events. The partial concurrency event corresponding to fig. 7 is combined as follows:

(A，D)、(D，A)；

(A，E)、(E，A)；

(B，D)、(D，B)；

(C，D)、(D，C)；

(H，D)、(D，H)；

(J，D)、(D，J)；……。

In one possible implementation, the above-described concurrent event combinations may be automatically generated by a distributed system time sequence testing tool, which is a tool that performs time sequence testing on a distributed system based on vector clock analysis. Optionally, the background of the distributed system timing test tool adopts python development, and the core part comprises data model modules (data model modules), algrothrms, generator modules and the like. The following describes each module.

1) model module

As shown in fig. 9, a schematic diagram of a model module in a distributed system timing test tool is schematically shown. The model module (model) 90 includes an initializer (init_builder) 91, an event builder (event_builder) 92, an interaction builder (interaction_builder) 93, a model builder (model_builder) 94, and a time axis builder (time_axis_builder) 95, among others.

The event_build constructor has the functions of acquiring Event id, acquiring Event description, acquiring Event attribute (Event in process, event sending, event receiving) and the like;

the interactive_builder constructor has the functions of acquiring interaction id, acquiring interaction description, interaction starting point event, interaction end point event and the like;

The timing_axis_builder time axis constructor has the functions of generating a time axis, adding events to the time axis, adding interactions to the time axis, acquiring the number of events of the time axis and the like;

the Model builder has functions of generating a Model, acquiring the Model, adding a time axis, adding an event, and the like.

2) Algorithms module

The Algorithms module contains an implementation of a vector clock generation algorithm. FIG. 10 is a schematic diagram illustrating the structure of an algorithm module in a distributed system timing test tool. The algorithm module (algorithm) 100 includes an initial unit (init_builder) 101, a Vector clock Test unit (test_vector_clock) 102, and a Vector clock unit (vector_clock) 103. The structure of the algorithm module (Algorithms) 100 shown in fig. 10 is merely a schematic illustration of a part of the structure, and represents the entire structure of the algorithm module 100.

Optionally, the Algorithms module includes functions of generating, merging, comparing, sorting, identifying concurrent events and the like of the vector clocks, and is a core module for generating the concurrent events.

3) Generator module

The Generator module is a module for converting the timing diagram into a vector clock, and mainly implements a vector clock base class and a basic function, and currently has generate_vclock (generating a vector clock), get_axis_nums (acquiring the number of time axes of the vector clock), get_event_nums (acquiring the number of events of the vector clock), and so on.

4) And a common module, a stop condition module, a statistics module and the like are also used for generating various concurrent event combinations of execution sequences corresponding to the concurrent events according to the corresponding strategies.

According to the embodiment of the application, the vector clock graph can be generated according to the time sequence graph of the distributed system, then all concurrent events can be automatically calculated and identified according to the vector clocks of all the events, the identified concurrent events can be ensured to be accurate and complete by adopting a vector clock analysis mode, and then all possible sequential execution sequence combinations of the constructed concurrent events are also complete and complete, partial scenes are not absent, the inheritance of the methodology is improved, and the accuracy and the completeness of the time sequence test can be improved.

And step 350, performing time sequence test on the concurrent event combination to obtain a test result.

The time sequence test is used for detecting abnormal concurrent events, and the test result is used for representing the execution condition of the concurrent event combination.

And (3) carrying out time sequence test on each concurrent event combination, and executing results of the concurrent events under different execution sequences. The concurrent events are the concurrent events identified by the system according to the clock information, and the process events which are the concurrent events can be successfully executed no matter what execution sequence is among the real concurrent events, so that a correct execution result is obtained. Therefore, by performing time sequence test on each concurrent event combination, the execution result of the concurrent event under different execution sequences can be obtained. If the execution result of a certain concurrent event combination is abnormal, it can prove that a logic problem exists between the process events corresponding to the concurrent event combination, and a developer is required to correct the logic problem.

In an exemplary embodiment, as shown in FIG. 4, the implementation of step 350 described above includes the following steps (351-352).

Step 351, obtaining at least one test case.

The test case is used for representing a use scene corresponding to the distributed system, and the test case can drive the at least two processes to run.

And step 352, performing time sequence test on the concurrent event combination based on at least one test case to obtain an execution result of the concurrent event combination in the time sequence test corresponding to the at least one test case.

Wherein the test results include execution results.

And based on at least one test case, carrying out time sequence test on the corresponding concurrent event combination of the first execution sequence and the corresponding concurrent event combination of the second execution sequence on each concurrent event to obtain an execution result of each concurrent event combination in the time sequence test corresponding to each test case.

In one example, as shown in FIG. 11, a schematic diagram of a test results presentation page is illustratively shown. In the test results presentation page 1110 shown in fig. 11, the test results corresponding to the concurrent event combinations on the various test cases are shown. The test cases in the test case column 1111 shown in fig. 11 are the cases in the blockchain scenario, including transaction uplink test cases, multi-transaction parallel uplink test cases, block test cases according to a high degree of search, transaction test cases according to a hash query, block test cases according to a hash query, contract test cases, and latest block test cases. Wherein, rpc.test_push_transaction.testPushtransaction (dev/rpc/test_push_transaction.json) [20210312190503091] is the name of the test case on the transaction; the name of the multi-transaction parallel uplink test case is rpc.test_push_transactions.TestPushtransactions (dev/rpc/test_push_transactions.json) [20210312190504093] rpc.testgetblock_byheight.testgetgetBayHeight (dev/pc/getblock_byheight.json) [20210312190504098] is the name of the test case according to the height-finding block; inquiring trade test cases according to the hash (hash) with the name rpc.testgettx _byhash.testgettrastnctionbyhash (dev/pc/gettx __ byhash.json) [20210312190504045]; the name of the test case is rpc.test_getblock_byhash.TestGetBlockByHash (dev/rpc/getblock_byhash.json) [20210312190504040]; inquiring names of contract test cases; rpc. test_query_contact. Testgetband (dev/rpc/querycontract_getband. Json) [20210312190504099]; the name of the latest block test case is rpc.testget_testreturnable_block.testlastlastretversibbleblock (dev/rpc/getcast_ireversibleblock.json) [20210312190504027]. The test results corresponding to each test case are all "pass" as shown in the test results column 1112 in the test results display page 1110.

In an exemplary embodiment, as shown in FIG. 4, the following steps (360-380) are further included after the step 350 described above.

Step 360, determining a process event corresponding to the target concurrent event combination under the condition that the test result represents that the execution result of the target concurrent event combination is abnormal.

The above-mentioned target concurrent event combination refers to a concurrent event combination with abnormal execution results, and if the target concurrent event combination, it is indicated that the process event corresponding to the target concurrent event combination is not a concurrent event, but is identified as a concurrent event before, it is indicated that the process event corresponding to the target concurrent event combination has a problem, so that the abnormal concurrent event needs to be found.

And step 370, obtaining abnormal concurrent event combinations based on the process events corresponding to the target concurrent event combinations.

Step 380, generating abnormal event positioning information corresponding to the abnormal concurrent event combination.

And forming an abnormal concurrent event combination by the process events corresponding to the target concurrent event combination, and generating abnormal event positioning information in a combined mode, so that a user can conveniently express the problem of the positioning system.

The technical solution provided in the embodiment of the present application may be applied to a distributed system timing test platform, as shown in fig. 12, which schematically illustrates an execution flow of the distributed system timing test platform. In the iterative testing period of the distributed system product, research and development engineers test the developed project to a software research and development test flow management platform, and then flow to a testing stage. Before acquiring the corresponding distributed system service binary package and configuration, a tester pulls the mirror image of the appointed version/appointed configuration from a mirror image warehouse uploaded in advance, starts a container, prepares the next test work, uploads a time sequence diagram corresponding to a distributed system algorithm, and a distributed system time sequence test platform generates a vector clock diagram according to the time sequence diagram and identifies concurrent events according to the vector clock diagram; after the distributed system time sequence test platform completes the identification and analysis of the concurrent events, generating event combinations of all test execution sequences corresponding to the concurrent events, then using an automatic use case to perform time sequence test on the event combinations, collecting monitoring indexes in each tested distributed node, and further generating a test report; problem localization can be performed based on the test report described above.

Based on the testing method provided by the embodiment of the application, a set of distributed system timing testing solutions can be determined, as shown in fig. 13, which schematically illustrates a block diagram of one distributed system timing testing solution. In the distributed system timing test solution shown in fig. 13, the method mainly includes three stages of distributed system test, timing test and release. In the distributed system test stage, after a developer develops a distributed product, the developed distributed product or project is tested to a software development test flow management platform, and then the software development test flow management platform distributes the software development test flow management platform; after the verification is passed, the test stage is entered. In the time sequence testing stage, the distributed system time sequence testing platform can construct a distributed system time sequence diagram according to event information in an application program package of a submitted distributed product, and generate a vector clock diagram according to the distributed system time sequence diagram so as to identify all concurrent events; and generating all test execution sequence combinations to perform time sequence test after identifying all concurrent events, generating a test report and issuing online. And a tester can find problems according to the published test report, inform a developer of the abnormality and quickly locate the problems.

In summary, according to the technical scheme provided by the embodiment of the application, by acquiring the logic connection relation between the events in different processes, the clock information of the events in different processes can be generated to determine the sequence relation between the events in different processes, so that the concurrent events in different processes can be automatically and accurately identified according to the clock information of the events, the manual judgment is not needed, and the identification accuracy of the concurrent events is improved; after the concurrent events are identified, the identified concurrent events can be combined, and the time sequence test is carried out on the concurrent event combination so as to detect abnormal concurrent events in different processes, so that test results which can represent the execution condition of the concurrent event combination in the time sequence test are obtained, the automatic modeling processing of the time sequence test is realized on the whole, the automation degree of the time sequence test is improved, the accuracy and the completeness of the time sequence test are improved, and the complexity of the time sequence test is reduced.

The following are device embodiments of the present application, which may be used to perform method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

Referring to fig. 14, a block diagram of a test apparatus according to an embodiment of the present application is shown. The device has the function of realizing the test method, and the function can be realized by hardware or by executing corresponding software by the hardware. The device may be a computer device or may be provided in a computer device. The apparatus 1400 may include: an event relationship acquisition module 1410, an event clock generation module 1420, a concurrent event determination module 1430, a concurrent event assembly module 1440, and an event assembly test module 1450.

The event relationship obtaining module 1410 is configured to obtain logical relationship information corresponding to process events in at least two processes, where the logical relationship information is used to characterize a logical connection relationship between the process events;

an event clock generation module 1420, configured to generate clock information corresponding to the process event based on the logical relationship information, where the clock information is used to characterize a sequential relationship corresponding to the process event in the at least two processes;

the concurrent event determining module 1430 is configured to determine a concurrent event set corresponding to the process event according to the clock information;

the concurrent event combination module 1440 is configured to perform event combination processing on the process event and a process event in the concurrent event set to obtain a concurrent event combination;

the event combination test module 1450 is configured to perform a timing test on the concurrent event combination to obtain a test result, where the timing test is used to detect an abnormal concurrent event, and the test result is used to characterize an execution condition of the concurrent event combination.

In an exemplary embodiment, the concurrent event combining module 1440 includes: the device comprises a concurrent event pair determining unit and a concurrent event pair arranging unit.

And the concurrent event pair determining unit is used for combining the process event with the process event in the concurrent event set to obtain a concurrent event pair.

The concurrent event pair arrangement unit is used for arranging the process events in the concurrent event pair to obtain a concurrent event combination of at least one execution sequence corresponding to the concurrent event pair.

In an exemplary embodiment, the concurrent event pair includes a first process event and a second process event, where the first process event and the second process event are concurrent events, and the concurrent event pair arrangement unit includes: a first arrangement subunit and a second arrangement subunit.

And the first arrangement subunit is used for arranging the first process event before the second process event to obtain a concurrent event combination of the concurrent event pair corresponding to the first execution sequence.

And the second arrangement subunit is used for arranging the first process event after the second process event to obtain a concurrent event combination of the concurrent event pair corresponding to the second execution sequence.

In an exemplary embodiment, the event relationship acquisition module 1410 includes: the device comprises a time axis determining unit, an event node determining unit, an interaction information determining unit, a vector edge determining unit and a timing diagram determining unit.

And the time axis determining unit is used for determining the time axis corresponding to each of the at least two processes.

And the event node determining unit is used for determining the event node corresponding to the process event on the time axis.

And the interactive information determining unit is used for determining an interactive event corresponding to the process event and a message transmission direction between the process event and the interactive event under the condition that the event type of the process event is a sending message event or a receiving message event.

And the vector edge determining unit is used for constructing a vector edge between the event node corresponding to the process event and the event node corresponding to the interaction event based on the message transmission direction.

And the time sequence diagram determining unit is used for obtaining a time sequence diagram corresponding to the process event based on the time axis, the event node and the vector edge, and the time sequence diagram is used for representing the logic relation information.

In an exemplary embodiment, the event clock generation module 1420 includes: the node position determining unit, the first time stamp determining unit, the preamble node determining unit, the preamble time stamp obtaining unit, the second time stamp determining unit and the node clock determining unit.

And the node position determining unit is used for determining time axis position information corresponding to the event node based on the time sequence diagram, wherein the time axis position information is used for representing the position arrangement order of the event node on a time axis.

And the first timestamp determining unit is used for determining first timestamp data corresponding to the event node on a first process according to the time axis position information, wherein the first process is a process corresponding to a time axis where the event node is located in the at least two processes.

And the preamble node determining unit is used for determining the preamble node corresponding to the event node in the time sequence diagram.

The preamble time stamp obtaining unit is configured to obtain time stamp data corresponding to the preamble node on a second process, where the second process refers to a process other than the first process in the at least two processes.

And the second timestamp determining unit is used for determining second timestamp data corresponding to the event node on the second process according to the timestamp data.

And the node clock determining unit is used for generating a vector clock corresponding to the process event based on the first timestamp data and the second timestamp data, and the vector clock is used for representing the clock information.

In an exemplary embodiment, the concurrency event determination module 1430 includes: the device comprises an event clock comparison unit, a concurrent event determination unit and an event set determination unit.

The event clock comparison unit is used for comparing the target vector clock corresponding to the target process event with vector clocks corresponding to other process events to obtain a vector clock comparison result; the target process event is any process event, and the other process events are process events except the target process event in the process events.

And the concurrent event determining unit is used for determining a target concurrent event corresponding to the target process event according to the vector clock comparison result, wherein the vector clock corresponding to the target concurrent event accords with the concurrent event clock condition with the target vector clock.

The event set determining unit is used for obtaining a target concurrent event set corresponding to the target process event based on the target concurrent event.

In an exemplary embodiment, the event combination test module 1450 includes: test case acquisition unit, time sequence test unit.

The test case acquisition unit is used for acquiring at least one test case.

And the time sequence test unit is used for carrying out the time sequence test on the concurrent event combination based on the at least one test case to obtain an execution result of the concurrent event combination in the time sequence test corresponding to the at least one test case.

Wherein the test result includes the execution result.

In an exemplary embodiment, the apparatus 1400 further comprises:

the abnormal event determining module is used for determining a process event corresponding to the target concurrent event combination under the condition that the test result represents that the execution result of the target concurrent event combination is abnormal;

the abnormal event group determining module is used for obtaining an abnormal concurrent event combination based on the process event corresponding to the target concurrent event combination;

the abnormal positioning information generation module is used for generating abnormal event positioning information corresponding to the abnormal concurrent event combination.

It should be noted that, in the apparatus provided in the foregoing embodiment, when implementing the functions thereof, only the division of the foregoing functional modules is used as an example, in practical application, the foregoing functional allocation may be implemented by different functional modules, that is, the internal structure of the device is divided into different functional modules, so as to implement all or part of the functions described above. In addition, the apparatus and the method embodiments provided in the foregoing embodiments belong to the same concept, and specific implementation processes of the apparatus and the method embodiments are detailed in the method embodiments and are not repeated herein.

Referring to fig. 15, a block diagram of a computer device according to an embodiment of the present application is shown. The computer device may be a server for performing the above-described test method. Specifically, the present invention relates to a method for manufacturing a semiconductor device.

The computer device 1500 includes a central processing unit (Central Processing Unit, CPU) 1501, a system Memory 1504 including a random access Memory (Random Access Memory, RAM) 1502 and a Read Only Memory (ROM) 1503, and a system bus 1505 connecting the system Memory 1504 and the central processing unit 1501. Computer device 1500 also includes a basic Input/Output system (I/O) 1506, and a mass storage device 1507 for storing an operating system 1513, application programs 1514, and other program modules 1515, which facilitate the transfer of information between the various devices within the computer.

The basic input/output system 1506 includes a display 1508 for displaying information and an input device 1509, such as a mouse, keyboard, etc., for the user to input information. Wherein the display 1508 and the input device 1509 are connected to the central processing unit 1501 via an input-output controller 1510 connected to the system bus 1505. The basic input/output system 1506 may also include an input/output controller 1510 for receiving and processing input from a number of other devices, such as a keyboard, mouse, or electronic stylus. Similarly, the input output controller 1510 also provides output to a display screen, a printer, or other type of output device.

The mass storage device 1507 is connected to the central processing unit 1501 through a mass storage controller (not shown) connected to the system bus 1505. The mass storage device 1507 and its associated computer-readable media provide non-volatile storage for the computer device 1500. That is, mass storage device 1507 may include a computer-readable medium (not shown) such as a hard disk or CD-ROM (Compact Disc Read-Only Memory) drive.

Computer readable media may include computer storage media and communication media without loss of generality. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes RAM, ROM, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory, electrically erasable programmable read-only memory), flash memory or other solid state memory technology, CD-ROM, DVD (Digital Video Disc, high density digital video disc) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices. Of course, those skilled in the art will recognize that computer storage media are not limited to the ones described above. The system memory 1504 and mass storage device 1507 described above may be collectively referred to as memory.

According to various embodiments of the present application, the computer device 1500 may also operate by being connected to a remote computer on a network, such as the Internet. That is, the computer device 1500 may be connected to the network 1512 via a network interface unit 1511 coupled to the system bus 1505, or alternatively, the network interface unit 1511 may be used to connect to other types of networks or remote computer systems (not shown).

The memory also includes a computer program stored in the memory and configured to be executed by the one or more processors to implement the above-described test methods.

In an exemplary embodiment, a computer readable storage medium is also provided, in which at least one instruction, at least one program, a set of codes, or a set of instructions is stored, which when executed by a processor, implement the above-described test method.

Alternatively, the computer-readable storage medium may include: ROM (Read Only Memory), RAM (Random Access Memory ), SSD (Solid State Drives, solid state disk), or optical disk, etc. The random access memory may include ReRAM (Resistance Random Access Memory, resistive random access memory) and DRAM (Dynamic Random Access Memory ), among others.

In an exemplary embodiment, a computer program product or a computer program is also provided, the computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions so that the computer device performs the above-described test method.

It should be understood that references herein to "a plurality" are to two or more. "and/or", describes an association relationship of an association object, and indicates that there may be three relationships, for example, a and/or B, and may indicate: a exists alone, A and B exist together, and B exists alone. The character "/" generally indicates that the context-dependent object is an "or" relationship. In addition, the step numbers described herein are merely exemplary of one possible execution sequence among steps, and in some other embodiments, the steps may be executed out of the order of numbers, such as two differently numbered steps being executed simultaneously, or two differently numbered steps being executed in an order opposite to that shown, which is not limited by the embodiments of the present application.

The foregoing description of the exemplary embodiments of the present application is not intended to limit the invention to the particular embodiments disclosed, but on the contrary, the intention is to cover all modifications, equivalents, alternatives, and alternatives falling within the spirit and scope of the invention.

Claims

1. A method of testing, the method comprising:

2. The method according to claim 1, wherein the performing event combination processing on the process event and the process event in the concurrent event set to obtain a concurrent event combination includes:

Combining the process event with the process event in the concurrent event set to obtain a concurrent event pair;

and arranging the process events in the concurrent event pairs to obtain a concurrent event combination of at least one execution sequence corresponding to the concurrent event pairs.

3. The method according to claim 2, wherein the concurrent event pair includes a first process event and a second process event, the first process event and the second process event are concurrent events, the arranging the process events in the concurrent event pair to obtain a concurrent event combination of at least one execution order corresponding to the concurrent event pair, including:

arranging the first process event before the second process event to obtain a concurrent event combination of the concurrent event pair corresponding to the first execution sequence;

and arranging the first process event after the second process event to obtain a concurrent event combination of the concurrent event pair corresponding to the second execution sequence.

4. The method according to claim 1, wherein the obtaining the logical relationship information corresponding to the process event in the at least two processes includes:

Determining time axes corresponding to the at least two processes respectively;

determining corresponding event nodes of the process event on the time axis;

determining an interaction event corresponding to the process event and a message transmission direction between the process event and the interaction event under the condition that the event type of the process event is a message sending event or a message receiving event;

based on the message transmission direction, constructing a vector edge between an event node corresponding to the process event and an event node corresponding to the interaction event;

and obtaining a time sequence diagram corresponding to the process event based on the time axis, the event node and the vector edge, wherein the time sequence diagram is used for representing the logic relation information.

5. The method of claim 4, wherein generating clock information corresponding to the process event based on the logical relationship information comprises:

determining time axis position information corresponding to the event node based on the time sequence diagram, wherein the time axis position information is used for representing the position arrangement order of the event node on a time axis;

determining first timestamp data corresponding to the event node on a first process according to the time axis position information, wherein the first process is a process corresponding to a time axis where the event node is located in the at least two processes;

Determining a corresponding preamble node of the event node in the time sequence diagram;

acquiring corresponding time stamp data of the preamble node on a second process, wherein the second process refers to other processes except the first process in the at least two processes;

determining second timestamp data corresponding to the event node on the second process according to the timestamp data;

and generating a vector clock corresponding to the process event based on the first timestamp data and the second timestamp data, wherein the vector clock is used for representing the clock information.

6. The method of claim 5, wherein determining the set of concurrent events corresponding to the process event according to the clock information comprises:

comparing the target vector clock corresponding to the target process event with vector clocks corresponding to other process events to obtain a vector clock comparison result; the target process event is any process event, and the other process events are process events except the target process event in the process events;

determining a target concurrent event corresponding to the target process event according to the vector clock comparison result, wherein the vector clock corresponding to the target concurrent event accords with a concurrent event clock condition with the target vector clock;

And obtaining a target concurrent event set corresponding to the target process event based on the target concurrent event.

7. The method according to any one of claims 1 to 6, wherein the performing a timing test on the concurrent event combination to obtain a test result includes:

acquiring at least one test case;

based on the at least one test case, performing the time sequence test on the concurrent event combination to obtain an execution result of the concurrent event combination in the time sequence test corresponding to the at least one test case;

wherein the test result includes the execution result.

8. The method according to any one of claims 1 to 6, further comprising:

determining a process event corresponding to the target concurrent event combination under the condition that the test result represents that the execution result of the target concurrent event combination is abnormal;

obtaining abnormal concurrent event combinations based on the process events corresponding to the target concurrent event combinations;

and generating abnormal event positioning information corresponding to the abnormal concurrent event combination.

9. A test apparatus, the apparatus comprising:

10. A computer device comprising a processor and a memory having stored therein at least one instruction, at least one program, code set or instruction set, the at least one instruction, at least one program, code set or instruction set being loaded and executed by the processor to implement the test method of any of claims 1 to 8.

11. A computer readable storage medium having stored therein at least one instruction, at least one program, code set, or instruction set, the at least one instruction, the at least one program, the code set, or instruction set being loaded and executed by a processor to implement the test method of any of claims 1 to 8.

12. A computer program product, characterized in that it comprises computer instructions stored in a computer-readable storage medium, from which computer instructions a processor of a computer device reads, which processor executes the computer instructions, so that the computer device executes to implement the test method according to any one of claims 1 to 8.