WO2019223062A1 - Method and system for processing system exceptions - Google Patents

Abstract

The present application is applied to the technical field of computers, and provides a method and a system for processing system exceptions. The method comprises: a communication terminal collecting system operating information thereof in real time, and reporting the system operating information to a monitoring node; the monitoring node generating a warning mail according to the system operating information and sending the warning mail to a central server, wherein the communication terminal having system exceptions and operation data for showing system exceptions of the communication terminal are recorded in the warning mail; the central server outputting the operating data in the warning mail to a pre-set processing solution database for matching, and acquiring processing scripts matched with the operating data; and the central server pushing the processing scripts to the communication terminal having system exceptions, wherein the processing scripts are executed automatically after being received by the communication terminal having system exceptions, and are used for processing the system exceptions. In the present application, automatic system operation and maintenance are realized, and timeliness of system operation and maintenance is also guaranteed.

Description

Method and system for processing system abnormality

This application claims priority from a Chinese patent application filed with the Chinese Patent Office on May 22, 2018, with application number 201810496049.9 and the invention name is "System Abnormal Processing Method and System", the entire contents of which are incorporated herein by reference .

Technical field

The present application relates to the field of computer technology, and in particular, to a method and system for processing system exceptions.

Background technique

With the continuous development of network technology, network equipment such as servers and gateways have been put into use on a large scale, and the capacity and topology complexity of the network have become larger and larger, which has led to the inevitable emergence of network systems during operation Various system exceptions.

At this stage, the system is mainly monitored by monitoring tools. Once a system abnormality occurs, the alarm information is usually transmitted to the relevant operation and maintenance personnel by mail or phone, and then the operation and maintenance personnel handle the system abnormality. However, many system failures occur repeatedly and the same processing methods exist. Existing system exception handling methods will cause a lot of tedious and repetitive work, reducing the operation and maintenance efficiency of the system.

technical problem

The embodiments of the present application provide a method and a system for processing system abnormalities, so as to solve the problem of low operation and maintenance efficiency of current network equipment when a system abnormality occurs.

Technical solutions

A first aspect of the embodiments of the present application provides a method for processing a system exception, including:

The communication terminal collects its system operation information in real time and reports the system operation information to the monitoring node;

The monitoring node generates an alarm email according to the system operation information, and sends the alarm email to a central server. The alarm email records a communication terminal where a system abnormality occurs and operating data indicating the system abnormality of the communication terminal. ;

The central server outputs the operation data in the alarm email to a preset processing solution database for matching, and obtains a processing script matching the operation data;

The central server pushes the processing script to the communication end where the system abnormality occurs, and the processing script is automatically executed after being received by the communication end where the system abnormality occurs, and is used to handle the system abnormality.

According to a second aspect of the embodiments of the present application, a system abnormality processing system is provided, including a central server and multiple communication terminals and multiple monitoring nodes deployed in a distributed manner.

The communication end is used to collect its system operation information in real time and report the system operation information to the monitoring node;

The monitoring node is configured to generate an alarm email according to the system operation information, and send the alarm email to the central server. The alarm email records a communication terminal where a system abnormality occurs and indicates the communication terminal system. Abnormal operating data;

The central server is configured to output the running data in the alarm email to a preset processing scheme database for matching, and obtain a processing script matching the running data;

The central server is further configured to push the processing script to the communication end where the system abnormality occurs, and the processing script is automatically executed after being received by the communication end where the system abnormality occurs, and is used to process the system abnormality.

Beneficial effect

In the embodiment of the present application, a central server is deployed in the existing network, and multiple monitoring nodes are deployed in a distributed manner. The original communication terminal in the network collects its system operation information in real time and reports the system operation information to Monitoring node. Based on the system operation information, the monitoring node generates an alarm email for the communication end where the system is abnormal and sends it to the central server, so that the central server matches the corresponding processing script in the preset processing plan database and pushes it to the communication end. Automatic processing. From the occurrence of system abnormality to the recovery of system abnormality, the entire process is automatically completed between the communication end, the monitoring node and the central server, realizing automatic system operation and maintenance, and also ensuring the timeliness of system operation and maintenance, saving operation and maintenance personnel Time and energy.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of a network topology architecture of a system exception handling system according to an embodiment of the present application; FIG.

FIG. 2 is an interaction flowchart of a method for processing a system exception according to an embodiment of the present application; FIG.

FIG. 3 is an implementation flowchart of a method for processing a system exception according to another embodiment of the present application; FIG.

4 is a flowchart of a method for processing a system exception according to another embodiment of the present application;

FIG. 5 is an implementation flowchart of a method for processing a system exception according to another embodiment of the present application; FIG.

FIG. 6 is an implementation flowchart of a method for processing a system exception according to another embodiment of the present application; FIG.

FIG. 7 is a flowchart of implementing a method for processing a system exception according to another embodiment of the present application; FIG.

FIG. 8 is a flowchart of implementing a method for processing a system exception according to another embodiment of the present application; FIG.

FIG. 9 is an interaction flowchart of a method for processing a system exception according to another embodiment of the present application; FIG.

FIG. 10 is a schematic diagram of a network node according to an embodiment of the present application.

Embodiments of the invention

In order to make the object, features, and advantages of the present application more obvious and understandable, the technical solutions in the embodiments of the present application will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present application. Obviously, The described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

FIG. 1 is a schematic diagram of a network topology architecture of a system exception handling system provided by an embodiment of the present application. For convenience of explanation, only a part related to this embodiment is shown.

Referring to FIG. 1, the system has a central server deployed in the network, and multiple communication terminals and multiple monitoring nodes are deployed in a distributed manner.

The communication end may be various network nodes that have been deployed in the network, such as network devices such as servers, gateways, and routers, and terminal devices such as computers, smart home appliances, and smart phones. In the embodiment of the present application, the system installed and running on the communication end is operated and maintained. Once a system abnormality occurs, the system abnormality is automatically restored based on the system abnormality processing method provided in the embodiment of the present application. For the communication end, it collects system operation information in real time during the system operation and reports the system operation information to the monitoring node.

The central server and the monitoring node are the devices deployed in the network in order to realize automatic recovery of system abnormalities in the embodiments of the present application. The monitoring nodes are deployed in the network in a distributed manner, and their equipment can be servers with high data processing capabilities. The monitoring nodes generate alarm emails based on the system operation information reported by the communication end, and send the alarm emails to the central server. It records the communication end where the system abnormality occurs and the operating data used to indicate the abnormality of the communication system. In the embodiment of the present application, one or more communication terminals may be deployed under a monitoring node, and each monitoring node is responsible for monitoring the system operation of the communication terminal deployed under it. Only one central server can be set in a network area, and the central server can simultaneously communicate with all monitoring nodes and communication terminals deployed in the network area. The processing server database is set on the central server. After receiving the alarm email reported by the monitoring node, the central server outputs the running data in the alarm email to the processing solution database for matching, obtains the corresponding processing script, and pushes the processing script to A communication end with a system exception. After receiving the processing script, the communication end automatically executes the processing script, thereby realizing the automatic recovery of the system exception.

Based on the embodiment shown in FIG. 1, further, each monitoring node is located under the same gateway as the communication end that reports system operation information to the monitoring node, so that the monitoring node can accurately and timely obtain the system operation information of the communication end. Moreover, the communication end reports its system operation information to the monitoring node under the same gateway, which is more assured in communication reliability and communication rate, and relatively improves the reporting efficiency of the system operation information, and facilitates operation and maintenance management.

Next, based on the embodiment shown in FIG. 1 of the present application, a method for processing a system exception provided by the embodiment of the present application will be described in detail. FIG. 2 shows an interaction flow of a method for processing a system exception provided by an embodiment of the present application. In this interaction flow, a communication entity involved in the interaction includes the foregoing central server, a monitoring node, and a communication end.

As shown in Figure 2, the method for processing the system exception includes:

S1: The communication terminal collects its system operation information in real time, and reports the system operation information to the monitoring node.

In the embodiment of the present application, a program for collecting system operation information is loaded in the communication terminal in advance, and the communication terminal implements real-time collection of system operation information through the preloaded program during the system operation. The collected system operation information includes, but is not limited to, business data processed by the system, system operation logs, basic resource usage of the communication end, database operation performance of the communication end, and middleware performance. After the system operation information is collected, the communication terminal reports the system operation information to the monitoring node in a regular or real-time manner.

As an embodiment of the present application, before S1, the communication end needs to determine a monitoring node that needs to report system operation information in advance. As shown in Figure 3:

S301: The communication terminal obtains a monitoring node list, and the monitoring node list records each gateway in the system and the monitoring nodes deployed under each of the gateways.

The monitoring node list is issued by the central server to each communication end, which records the information of each gateway in the system and the monitoring nodes deployed under each gateway. In the monitoring node list, each gateway and each monitoring node can use their IP addresses Display. The monitoring node list is maintained by the central server. When the content recorded in it is changed, the central server re-delivers it to the communication end. After receiving the new monitoring node list, the communication end updates the locally stored monitoring node list.

S302: The communication end finds its gateway in the monitoring node list.

As described in the above embodiment, preferably, each monitoring node is located under the same gateway as the communication end to which the system operation information is reported. Therefore, in this embodiment, the communication end first looks for the communication in the monitoring node list. The gateway where the end is located.

S303: The communication terminal determines the found monitoring node deployed under the gateway as the monitoring node that needs to report the system operation information.

After finding the gateway where the communication end is located in the monitoring node list, the communication end selects any monitoring node deployed under the gateway, and determines the monitoring node as the monitoring node to which the communication end needs to report system operation information.

In the embodiment corresponding to FIG. 3, the communication end reports its system operation information to a monitoring node under the same gateway, which is more assured in communication reliability and communication rate, and relatively improves the efficiency of reporting system operation information, and Convenient for operation and maintenance management.

After S303, as shown in FIG. 4, it further includes:

S304: The communication end records all the monitoring nodes deployed under the gateway found.

The communication end records all the monitoring nodes deployed under its gateway. For example, if five monitoring nodes are deployed under the gateway where the communication terminal is located, then in addition to configuring one of the monitoring nodes as a monitoring node that needs to report system operating information, the communication terminal addresses the other four monitoring nodes with address information and node identifiers. Make a record.

S305: If the system operation information reporting failure is detected, the communication end selects another monitoring node under the found gateway as the monitoring node that needs to report the system operation information.

After the communication end reports the system operation information to the monitoring node, it usually receives a response that the system operation information received by the monitoring node is successfully received. If the response is not received within a certain period of time, the communication end defaults to failing to report the system operation information. Then, at this time, the communication end selects another monitoring node under its gateway to report system operation information according to the information recorded in S304. The embodiment corresponding to FIG. 4 considers the possibility of failure of a monitoring node or a communication link, and establishes a backup reporting mechanism for the smooth reporting of system operation information, which effectively guarantees the timeliness of system operation and maintenance.

S2: The monitoring node generates an alarm email according to the system operation information, and sends the alarm email to the central server. The alarm email records a communication terminal where a system abnormality occurs and a system for indicating the communication terminal. Abnormal operating data.

The monitoring node analyzes the system operation information reported by each communication terminal to monitor the program abnormality or business data abnormality in the system run by each communication terminal, and generates an alarm email based on the related information of the system abnormality based on the monitoring result, and sends it to the center server. The alarm email mainly describes the device identification or network address of the communication end where the system abnormality occurs, and writes the operating data used to indicate the system abnormality of the communication end.

As an embodiment of the present application, before S2, the monitoring node may foretell the establishment of a system normal operation model, so as to import system operation information into the model, thereby determining whether the system on the corresponding communication end operates normally. As shown in Figure 5:

S501: within a preset time period, the monitoring node collects the system operation information of different communication terminals.

The monitoring node can collect the system operation information of each communication end in advance and store it in the operation information set for subsequent modeling and analysis.

S502: The monitoring node performs clustering on the collected system operation information to obtain multiple cluster sets.

The monitoring node uses a clustering algorithm, such as the CURE clustering algorithm, to cluster the collected system operation information to obtain multiple cluster sets, and the system operation information in each cluster set has the same or similar data characteristics.

S503: The monitoring node marks a cluster set for indicating that the system operates normally in the multiple cluster sets.

According to the preset experience values, the monitoring nodes mark clusters that indicate the normal operation of the system in the multiple cluster sets generated. The system operation information in these cluster sets can indicate that no system abnormality has occurred on the corresponding communication end.

As an embodiment of the present application, the implementation of S503 is shown in FIG. 6:

S601: The monitoring node arranges the multiple cluster sets in descending order according to the size of the cluster.

After clustering, the amount of system operation information gathered in each cluster set is different, so first, the monitoring node will generate multiple clusters according to the size of the cluster, that is, according to the amount of system operation information gathered in the cluster set. Sets are sorted in descending order.

S602: The monitoring node reads a preset scaling parameter, where the scaling parameter is used to indicate that the number of normal communication terminals of the system at the same time accounts for all communication terminals.

The proportionality parameter is determined by empirical values or previous system operating conditions. It is used to indicate that at the same time, the normal communication end of the system accounts for the proportion of all communication ends, that is, the system operation information used to indicate that the system is operating normally throughout the operation. Percentage of information concentration.

S603: The monitoring node marks the clusters arranged in the top N positions as clusters for indicating that the system operates normally based on the preset scaling parameters.

After obtaining the preset scale parameters, the monitoring nodes will be ranked in the top N according to the preset scale parameters, the amount of system operation information reported by the communication end in the current statistical time period, and the amount of system operation information in each cluster set. A cluster of bits is marked as a cluster to indicate that the system is operating normally. The ratio of the sum of the system operation information in the labeled cluster set to the sum of the system operation information in all cluster sets is approximately equal to a preset ratio parameter.

In the embodiment corresponding to FIG. 6, the system operation information is filtered according to the experience value and the clustering algorithm, and the system operation information used to indicate the normal operation of the system is determined from it, which is used for the subsequent system normal operation modeling. deal with.

S604: The monitoring node generates a normal operation model of the system based on the clusters marked, and the normal operation model of the system is used by the monitoring node to determine whether the system operation information reported by the communication terminal indicates that the communication terminal The system is running normally.

For the marked clusters, the monitoring node obtains the system operation information in it, and performs modeling to generate the normal operation model of the system. The normal operation model of the system can be established based on neural networks. The system operation information represented by the clusters is used as input samples, and the system operation status represented by the system operation information, that is, the system normal or system abnormality is used as the output result for model training. After training, the model is used to determine whether the system operation information reported by the communication end can indicate that the communication end system is operating normally.

S3: The central server outputs the operation data in the alarm email to a preset processing scheme database for matching, and obtains a processing script matching the operation data.

In the embodiment of the present application, the central server can only set a timing task in its background to periodically obtain the alarm email generated by the monitoring node. In the alarm email, the operating data used to indicate the abnormality of the communication end system can be attached to the email as a text attachment, or it can be reflected in the form of the email body. The central server parses the relevant text content in the alarm email sent by the monitoring node, including segmenting the text content, finding out a character string that can characterize the system's operating indicators, and reading the corresponding data of the character string to convert the text information into A data table used to characterize the operating status of the system. The key names in this data table are character strings that can characterize system operation indicators, and the key values are the corresponding data for each character string. The character string capable of characterizing the system operation index includes, but is not limited to, a server number, a server address, a time when a system abnormality occurs, a description of a system abnormality, an operating parameter when the system is abnormal, and the like.

A processing plan database is created in the central server, and the processing plan database is created at least before executing S3, as shown in Figure 7:

S701: The central server enters a configuration mode.

After the configuration mode is triggered, the central server presents a configurable page to the operation and maintenance user, and the operation and maintenance personnel can configure the processing plan database on the configurable page.

S702: In the configuration mode, the central server receives the characteristic parameters and corresponding processing scripts input by the operation and maintenance user for describing system abnormalities.

On the configurable page, operation and maintenance users enter characteristic parameters that describe system exceptions and corresponding processing scripts. For each type of system abnormality, the character strings used to characterize the system's operating indicators will correspond to different values, and these different value character strings can form the characteristic parameters used to describe the system abnormality.

S703: The central server stores feature parameters input by the operation and maintenance user in association with corresponding processing scripts, and the feature parameters are used by the central server to match with the operation data.

The central server stores the characteristic parameters entered by the operation and maintenance user on the configurable page to describe a certain type of system abnormality, and stores it with the processing script used to recover the type of system abnormality. In this way, in the analysis of the alarm email, After the operating data used to indicate the abnormality of the system at the communication end, the character string used to characterize the system operation index in the operating data and the character string number corresponding to the character string used to characterize the system operation index in the characteristic parameters are matched, so that Can determine the type of system exception, and further determine the processing script used to handle this type of system exception.

S4: The central server pushes the processing script to the communication end where the system abnormality occurs, and the processing script is automatically executed after being received by the communication end where the system abnormality occurs, and is used to handle the system abnormality.

After the central server determines in its processing plan database that it can match the processing script of the communication system exception, it pushes the processing script to the communication terminal according to the relevant information of the communication terminal where the system abnormality is described in the alarm email, such as the network address of the communication terminal . After receiving the script pushed by the central server, the communication end will automatically execute the processing script to recover the system exception.

Further, at the communication end, the priority of processing scripts is set to ensure the timely recovery of system exceptions, as shown in Figure 8:

S801: After detecting the processing script pushed by the central server, the communication end automatically creates an execution thread of the processing script.

At the communication end, the push of the processing script is used as a trigger condition in advance, that is, once it is detected that the central server has pushed the processing script, a thread creation action is triggered, and an execution thread for the processing script is automatically created locally.

S802: The communication end sets the execution thread with the highest priority to automatically execute the processing script first.

After the execution thread is created, the communication end sets the priority of the execution thread to the highest. In this way, the other threads that the communication end has previously created are lower than the priority of the execution thread, and the communication end will run immediately. The execution thread is used to execute a processing script, thereby realizing timely recovery of system exceptions.

Further, as an embodiment of the present application, after the system exception is processed, as shown in FIG. 9:

S5: The communication end feeds back the execution log of the processing script to the central server after the system exception processing is successful.

During the execution of the processing script, the communication end records the execution process, generates an execution log, and feeds back to the central server the result of the successful system exception processing and the execution log.

S6: The central server performs statistical analysis on the received execution logs at preset time intervals, and generates a prediction report on the system operating status according to the results of the statistical analysis.

The central server records the communication end that successfully handles the system abnormality according to the received execution log, and statistically analyzes the system abnormality of the communication end and the corresponding processing result at intervals, and generates the system operating status based on the results of the statistical analysis Forecast report to better help operation and maintenance personnel understand the operation of the system, better improve system functions, and improve system stability.

Based on the method for recovering system abnormalities described above, automatic recovery of system abnormalities can be achieved. For example, when a communication terminal in the system has an abnormal thread blocking, the monitoring node analyzes the system operation information reported by the communication terminal to confirm that the communication terminal has a system abnormality, so it sends an alarm email to the central server. The central server analyzes the text of the alarm email, and imports the parsing result into the processing solution database for matching to find the processing script for processing thread blocking, and pushes the processing script to the communication terminal according to the address of the communication terminal in the alarm email. The communication end automatically executes the processing script to complete system exception recovery.

It should be understood that the size of the sequence numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

FIG. 10 is a schematic diagram of a network node according to an embodiment of the present application. Here, the network node may be a central server, a communication end, or a monitoring node in FIG. 1. As shown in FIG. 10, the network node 10 of this embodiment includes a processor 100, a memory 101, and computer-readable instructions 102 stored in the memory 101 and executable on the processor 100. When the processor 100 executes the computer-readable instructions 102, the steps in the embodiment of a method for restoring a system abnormality corresponding to each network node are implemented. For example, for a communication end, the processor 100 executes In step S1, for a monitoring node, the processor 100 executes step S2 shown in FIG. 2, and for a central server, the processor 100 executes steps S3 and S4 shown in FIG.

Exemplarily, the computer-readable instructions 102 may be divided into one or more modules / units, the one or more modules / units are stored in the memory 101 and executed by the processor 100, To complete this application. The one or more modules / units may be an instruction segment of a series of computer-readable instructions capable of performing a specific function, and the instruction segment is used to describe an execution process of the computer-readable instruction 102 in its corresponding network node.

The processor 100 may be a central processing unit (Central Processing Unit (CPU), or other general-purpose processors, Digital Signal Processors (DSPs), and application-specific integrated circuits (Applications) Specific Integrated Circuit (ASIC), off-the-shelf Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory 101 may be an internal storage unit of a corresponding network node, such as a hard disk or a memory at a communication end. The memory 101 may also be an external storage device of a corresponding network node, such as a plug-in hard disk equipped on a communication end, a smart memory card (Smart Media Card, SMC), Secure Digital (SD) card, Flash Card, etc. Further, the memory 101 may further include both an internal storage unit of a corresponding network node and an external storage device. The memory 101 is configured to store the computer-readable instructions and other programs and data required by the server. The memory 101 may also be used to temporarily store data that has been output or is to be output.

This application implements all or part of the processes in the methods of the above embodiments, and may also be completed by computer-readable instructions instructing related hardware. The computer-readable instructions may be stored in a computer-readable storage medium. The computer may When the read instruction is executed by the processor, the steps of the foregoing method embodiments can be implemented. The computer-readable instructions include computer-readable instruction codes, and the computer-readable instruction codes may be in a source code form, an object code form, an executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer-readable instructions, a recording medium, a U disk, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signals, telecommunication signals, and software distribution media. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdictions. For example, in some jurisdictions, the computer-readable medium Excludes electric carrier signals and telecommunication signals.

The above-mentioned embodiments are only used to describe the technical solution of the present application, but not limited thereto. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still implement the foregoing implementations. The technical solutions described in the examples are modified, or some of the technical features are equivalently replaced; and these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the application, and should be included in Within the scope of this application.

Claims (20)

A method for processing system exceptions, which comprises: The communication end collects system operation information in real time and reports the system operation information to a monitoring node; The monitoring node generates an alarm email according to the system operation information, and sends the alarm email to a central server. The alarm email records a communication terminal where a system abnormality occurs and operating data indicating the system abnormality of the communication terminal. ; The central server outputs the operation data in the alarm email to a preset processing solution database for matching, and obtains a processing script matching the operation data; The central server pushes the processing script to the communication end where the system abnormality occurs, and the processing script is automatically executed after being received by the communication end where the system abnormality occurs, and is used to handle the system abnormality. The method for processing a system abnormality according to claim 1, wherein before the communication terminal collects system operation information in real time and reports the system operation information to the monitoring node, further comprising: The communication terminal obtains a list of monitoring nodes, where each gateway in the system and the monitoring nodes deployed under each of the gateways are recorded in the monitoring node list; The communication end finds its gateway in the list of monitoring nodes; The communication terminal determines the found monitoring node deployed under the gateway as the monitoring node that needs to report the system operation information. The method for processing a system abnormality according to claim 2, wherein after the communication end determines that the monitored monitoring node deployed under the gateway is a monitoring node that needs to report the system operation information, further comprising: The communication end records all the monitoring nodes deployed under the gateway found; If the system operation information reporting failure is detected, the communication end selects another monitoring node under the found gateway as the monitoring node that needs to report the system operation information. The method for processing a system abnormality according to claim 1, before the generating, by the monitoring node, an alarm email according to the system operation information, further comprising: Within a preset time period, the monitoring node collects the system operation information of different communication terminals; The monitoring node performs clustering on the collected system operation information to obtain multiple cluster sets; The monitoring node marks a cluster set for indicating that the system operates normally in the multiple cluster sets; The monitoring node generates a normal operation model of the system based on the marked clusters, and the system normal operation model is used by the monitoring node to determine whether the system operation information reported by the communication end indicates that the communication end system is normal. run. The method for processing a system abnormality according to claim 4, wherein the monitoring node marks the cluster set in the multiple cluster sets to indicate that the system operates normally, comprising: The monitoring node arranges the plurality of cluster sets in descending order according to the size of the cluster; The monitoring node reads a preset proportion parameter, and the proportion parameter is used to indicate that the number of normal communication terminals of the system at the same time accounts for the proportion of all communication terminals; The monitoring node marks the clusters arranged in the top N positions as clusters used to indicate that the system operates normally based on the preset scaling parameters. The method for processing a system abnormality according to claim 5, wherein the monitoring node marks the clusters ranked in the top N positions as indicating that the system is operating normally based on the preset scaling parameters. Clusters, including: The monitoring node will be ranked in the top N according to the preset scale parameter, the amount of the system operation information reported by the communication terminal in the current statistical time period, and the amount of the system operation information in each cluster set. The cluster of bits is marked as a cluster to indicate that the system is operating normally. The method for processing a system abnormality according to claim 1, wherein the central server outputs the characteristic data in the alarm email to a preset processing scheme database for matching to obtain the characteristic data Before the processing script corresponding to the data, it also includes: The central server enters a configuration mode; In the configuration mode, the central server receives characteristic parameters and corresponding processing scripts input by the operation and maintenance user for describing system abnormalities; The central server stores the characteristic parameters input by the operation and maintenance user in association with the corresponding processing script, and the characteristic parameters are used by the central server to match with the operation data. The method for processing a system exception according to claim 1, further comprising: After the communication end successfully handles the system exception, the communication end feeds back the execution log of the processing script to the central server; Performing statistical analysis on the execution log received by the central server every preset time interval; The central server generates a prediction report on a system operating condition according to a result of statistical analysis. The method for processing a system exception according to claim 1, further comprising: After detecting the processing script pushed by the central server, the communication end automatically creates an execution thread of the processing script; The communication end sets the execution thread with the highest priority to automatically execute the processing script first. The method for processing a system abnormality according to claim 1, wherein the alarm email records a device identifier or a network address of a communication end where the system abnormality occurs. A system abnormality processing system is characterized in that it includes a central server and multiple communication terminals and multiple monitoring nodes in a distributed deployment. The communication end is used to collect its system operation information in real time and report the system operation information to the monitoring node; The monitoring node is configured to generate an alarm email according to the system operation information, and send the alarm email to the central server. The alarm email records a communication terminal where a system abnormality occurs and indicates the communication terminal system. Abnormal operating data; The central server is configured to output the running data in the alarm email to a preset processing scheme database for matching, and obtain a processing script matching the running data; The central server is further configured to push the processing script to the communication end where the system abnormality occurs, and the processing script is automatically executed after being received by the communication end where the system abnormality occurs, and is used to process the system abnormality. The processing system according to claim 11, wherein the monitoring node is located under the same gateway as the communication end to which the system operation information is reported. The processing system according to claim 11, wherein before the communication terminal collects its system operation information in real time and reports the system operation information to the monitoring node, the communication terminal is further configured to: Obtaining a list of monitoring nodes, where each gateway in the system and the monitoring nodes deployed under each of the gateways are recorded; Find the gateway where the monitoring node is located; The found monitoring node deployed under the gateway is determined as the monitoring node that needs to report the system operation information. The processing system according to claim 13, wherein after the communication end determines that the monitoring node deployed under the found gateway is a monitoring node that needs to report the system operation information, the communication end further Used for: Record all the monitoring nodes deployed under the gateway found; If the system operation information reporting failure is detected, another monitoring node is selected under the found gateway as the monitoring node that needs to report the system operation information. The processing system according to claim 11, wherein before the monitoring node generates an alarm email according to the system operation information, the monitoring node is further configured to: Collecting the system operation information of the different communication terminals within a preset time period; Clustering the collected system operation information to obtain multiple cluster sets; Marking in the plurality of cluster sets a cluster set for indicating that the system operates normally; A normal operation model of the system is generated based on the marked clusters, and the normal operation model of the system is used by the monitoring node to determine whether the system operation information reported by the communication end indicates that the communication end system is operating normally. The processing system according to claim 15, wherein the monitoring node is specifically configured to: Arranging the multiple cluster sets in descending order according to the size of the clusters; Read a preset proportion parameter, which is used to indicate the proportion of the number of all communication terminals that are normal communication terminals of the system at the same time; Based on the preset scale parameters, the clusters arranged in the top N positions are marked as clusters used to indicate that the system is operating normally. The processing system according to claim 16, wherein the monitoring node is specifically configured to: According to the preset scale parameter, the number of the system operation information reported by the communication terminal in the current statistical time period, and the number of the system operation information in each cluster set, the first Clusters are marked as clusters to indicate that the system is operating normally. The processing system according to claim 11, wherein the central server outputs the feature data in the alarm email to a preset processing scheme database for matching, and obtains a feature corresponding to the feature data. Before processing the script, the central server is also used to: Enter configuration mode; In the configuration mode, receive characteristic parameters and corresponding processing scripts input by the operation and maintenance user for describing system abnormalities; The characteristic parameters input by the operation and maintenance user are stored in association with corresponding processing scripts, and the characteristic parameters are used by the central server to match with the operation data. The processing system according to claim 11, wherein: The communication end is further configured to feed back the execution log of the processing script to the central server after the system exception processing is successful; The central server is further configured to perform statistical analysis on the execution logs received at preset time intervals; The central server is further configured to generate a prediction report on a system operating status according to a result of statistical analysis. The processing system according to claim 11, wherein: After detecting the processing script pushed by the central server, the communication end is further configured to automatically create an execution thread of the processing script; The communication end is further configured to set the execution thread with the highest priority to automatically execute the processing script first.