CN114780341A

CN114780341A - Log processing method and log processing device

Info

Publication number: CN114780341A
Application number: CN202210423621.5A
Authority: CN
Inventors: 武文轩; 谢伟; 程鹏; 王磊
Original assignee: Industrial and Commercial Bank of China Ltd ICBC
Current assignee: Industrial and Commercial Bank of China Ltd ICBC
Priority date: 2022-04-21
Filing date: 2022-04-21
Publication date: 2022-07-22

Abstract

The application provides a log processing method and a log processing device, and relates to the field of cloud computing. The method comprises the following steps: generating logs in different log formats for the application service; converting the logs in different log formats into logs in a preset format; determining a processing strategy of the log in the preset format according to the log type of the log in the preset format and a first corresponding relation, wherein the first corresponding relation comprises a plurality of groups of corresponding relations, each group of corresponding relation in the plurality of groups of corresponding relations is used for indicating the corresponding relation between the log type and the processing strategy, the processing strategy comprises an acquisition strategy and a sending strategy, the acquisition strategy is used for acquiring the log in the preset format, and the sending strategy is used for sending the acquired information for viewing; and processing the log in the preset format according to the processing strategy. The method can improve the log processing efficiency.

Description

Log processing method and log processing device

Technical Field

The present application relates to the field of cloud computing, and in particular, to a log processing method and a log processing apparatus.

Background

With the continuous development of container technology, the container technology is widely applied to the field of cloud computing. When the cloud server runs the application service by using the container technology, a user can generate logs in different formats by selecting different plug-ins. When a user views the logs, a log collection tool adapted to the log format needs to be selected according to the log format to collect data in the logs, and when the logs have different log types, the user needs to determine output strategies of the logs according to the different log types to output the collected data for analysis and processing.

In the above process, too many manual interventions exist, and a user needs to determine the format of the log, the log collection tool adapted to the format, and determine the output strategy, so that the log processing process is complicated, and the log processing efficiency is affected.

Therefore, a log processing method is needed to improve the efficiency of log processing.

Disclosure of Invention

The application provides a log processing method and a log processing device, which can improve log processing efficiency.

In a first aspect, the present application provides a log processing method, including: generating logs in different log formats for the application service; converting the logs in different log formats into logs in a preset format; determining a processing strategy of the log in the preset format according to the log type of the log in the preset format and a first corresponding relation, wherein the first corresponding relation comprises a plurality of groups of corresponding relations, each group of corresponding relation in the plurality of groups of corresponding relations is used for indicating the corresponding relation between the log type and the processing strategy, the processing strategy comprises an acquisition strategy and a sending strategy, the acquisition strategy is used for acquiring the log in the preset format, and the sending strategy is used for sending the acquired information for viewing; and processing the log in the preset format according to the processing strategy.

The log processing method provided by the application can automatically convert logs in different log formats into logs in a preset format for the application service, reduces the step of selecting an acquisition tool according to the format of the logs, determines the processing strategy of the logs in the preset format according to the log type and the first corresponding relation of the logs in the preset format, reduces the step of manually determining the processing strategy, reduces the workload of a user, improves the automation level, and is beneficial to improving the log processing efficiency.

With reference to the first aspect, in certain implementation manners of the first aspect, before determining the processing policy of the log in the preset format according to the log type of the log in the preset format and the first corresponding relationship, the method further includes: classifying the logs in the preset format according to the log types of the logs in the preset format to obtain at least one category of logs; the determining the processing strategy of the log in the preset format according to the log type of the log in the preset format and the first corresponding relation comprises the following steps: determining a processing strategy of the log of each category in the logs of the at least one category according to the log type corresponding to the log of the at least one category and the first corresponding relation; the processing the log according to the processing policy includes: and processing the logs of each category according to the processing strategy of the logs of each category.

According to the log processing method, the logs in the preset format are classified, then different processing strategies are used for the logs in different types, the same processing strategy is used for the logs in the same type, the processing efficiency of each log does not need to be judged one by one, and the log processing efficiency can be further improved.

With reference to the first aspect, in certain implementations of the first aspect, the method further includes: and if the log type of the first log in the logs with the preset format is a preset type, determining the processing strategy of the first log as a high-priority processing strategy.

According to the log processing method, the high priority is set for the preset type of logs, important logs can be processed preferentially, the user requirements can be met, and the flexibility of log processing is improved.

With reference to the first aspect, in certain implementations of the first aspect, the method further includes: monitoring the processing condition of the first log to obtain monitoring data; and adjusting the processing strategy of the first log according to the monitoring data.

According to the log processing method, the processing condition of the first log which is processed preferentially can be monitored in real time, so that the processing strategy is optimized, and the processing strategy can be more suitable for application scenes.

With reference to the first aspect, in some implementation manners of the first aspect, the processing policy further includes an information filtering policy and a monitoring policy, where the information filtering policy is used to filter information in the log, and the monitoring policy is used to monitor the acquisition policy and the sending policy to adjust the acquisition policy and the sending policy.

With reference to the first aspect, in certain implementations of the first aspect, the log type of the log in the preset format includes at least one of: transaction type logs, business logs, monitoring logs, system logs, or buried point analysis logs.

In a second aspect, the present application provides a log processing apparatus, which includes a generating module and a processing module. The generation module is used for: generating logs in different log formats for the application service; the processing module is used for converting the logs in different log formats into logs in a preset format; determining a processing strategy of the log in the preset format according to the log type of the log in the preset format and a first corresponding relation, wherein the first corresponding relation comprises multiple groups of corresponding relations, each group of corresponding relation in the multiple groups of corresponding relations is used for indicating the corresponding relation between the log type and the processing strategy, the processing strategy comprises an acquisition strategy and a sending strategy, the acquisition strategy is used for acquiring the log in the preset format, and the sending strategy is used for sending the acquired information for viewing; and processing the log with the preset format according to the processing strategy.

With reference to the second aspect, in certain implementations of the second aspect, the processing module is further configured to: classifying the logs in the preset format according to the log types of the logs in the preset format to obtain at least one category of logs; determining a processing strategy of the log of each category in the logs of the at least one category according to the log type corresponding to the log of the at least one category and the first corresponding relation, wherein the first corresponding relation comprises the log type corresponding to the log of the at least one category; and processing the logs of each category according to the processing strategy of the logs of each category.

With reference to the second aspect, in certain implementations of the second aspect, the processing module is further configured to: and if the log type of the first log in the logs with the preset format is a preset type, determining the processing strategy of the first log as a high-priority processing strategy.

With reference to the second aspect, in certain implementations of the second aspect, the processing module is further configured to: monitoring the processing condition of the first log to obtain monitoring data; and adjusting the processing strategy of the first log according to the monitoring data.

With reference to the second aspect, in some implementation manners of the second aspect, the processing policy further includes an information filtering policy and a monitoring policy, the information filtering policy is used to filter information in the log, and the monitoring policy is used to monitor the acquisition policy and the sending policy to adjust the acquisition policy and the sending policy.

With reference to the second aspect, in some implementations of the second aspect, the log type of the log in the preset format includes at least one of: transaction type logs, service logs, monitoring logs, system logs, or buried point analysis logs.

In a third aspect, the present application provides a log processing apparatus comprising a processor and a memory. The processor is configured to read instructions stored in the memory to perform the method of any one of the possible implementations of the first aspect.

Optionally, there are one or more processors and one or more memories.

Alternatively, the memory may be integrated with the processor, or provided separately from the processor.

In a specific implementation process, the memory may be a non-transient memory, such as a Read Only Memory (ROM), which may be integrated on the same chip as the processor, or may be separately disposed on different chips.

The log processing apparatus in the third aspect may be a chip, the processor may be implemented by hardware or may be implemented by software, and when implemented by hardware, the processor may be a logic circuit, an integrated circuit, or the like; when implemented in software, the processor may be a general-purpose processor implemented by reading software code stored in a memory, which may be integrated with the processor, located external to the processor, or stand-alone.

In a fourth aspect, the present application provides a computer-readable storage medium having a computer program (also referred to as code, or instructions) which, when run on a computer, causes the computer to perform the method of any one of the possible implementations of the first aspect.

In a fifth aspect, the present application provides a computer program product comprising: computer program (also called code, or instructions), which when executed, causes a computer to perform the method of any of the possible implementations of the first aspect.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.

Fig. 1 is a schematic flowchart of a log processing method according to an embodiment of the present application;

FIG. 2 is a schematic block diagram of a log collection tool according to an embodiment of the present application;

FIG. 3 is a schematic flow chart diagram of another log processing method according to an embodiment of the present application;

fig. 4 is a schematic block diagram of a log processing apparatus according to an embodiment of the present application;

fig. 5 is a schematic block diagram of another log processing apparatus according to an embodiment of the present application.

With the above figures, there are shown specific embodiments of the present application, which will be described in more detail below. The drawings and written description are not intended to limit the scope of the inventive concepts in any manner, but rather to illustrate the concepts of the application by those skilled in the art with reference to specific embodiments.

Detailed Description

The technical solution in the present application will be described below with reference to the accompanying drawings. The log processing method and the log processing apparatus provided by the present application may be used in the cloud computing field, and may also be used in any field other than the cloud computing field. The log processing method and the log processing device provided by the application are not limited to the application field.

For the convenience of understanding the embodiments of the present application, the related terms in the embodiments of the present application will be described first.

1. Application service

Application services refer to contractual services that provide deployment, provisioning, management, and lease access to applications for centrally managed organizations.

2. Container technology

The container technique can effectively divide the resources of a single operating system into isolated groups so as to better balance conflicting resource usage requirements among the isolated groups.

3. Acquisition configuration

The acquisition configuration is used to formulate acquisition rules that are reusable in the solution, which determine the source, content, and processing of the acquisition. In one scheme, one acquisition rule must be selected, and the acquisition rule cannot be permanently effective and can be modified according to conditions.

The container technology is widely applied to the field of cloud computing, can directly run all business applications on an operating system of a physical host, and can directly read and write a disk. The business applications can be isolated by computing, storing and network resource namespaces, and a logically independent container operating system is formed for each application.

When the cloud server runs the application service by using the container technology, a user can generate logs in different formats by selecting different plug-ins. When a user views the logs, a log acquisition tool adaptive to the log format is selected according to the log format to acquire data in the logs, and when the logs have different log types, the user needs to determine output strategies of the logs according to the different log types to output the acquired data for analysis and processing, so that resource contention is avoided, and the accuracy and the integrity of the data are not affected.

In the above process, too many manual interventions exist, and a user needs to determine the format of the log, the log acquisition tool adapted to the format, and the output policy, so that the log processing process is complicated, and the log processing efficiency is affected.

In view of this, embodiments of the present application provide a log processing method and a log processing apparatus, which can improve log processing efficiency.

Before describing the log processing method and the log processing apparatus provided in the embodiments of the present application, the following description is made.

In the embodiments of the present application, "at least one" means one or more, "a plurality" means two or more. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a alone, A and B together, and B alone, wherein A and B may be singular or plural. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. "at least one of the following" or similar expressions refer to any combination of these items, including any combination of the singular or plural items. For example, at least one (one) of a, b, and c, may represent: a, or b, or c, or a and b, or a and c, or b and c, or a, b and c, wherein a, b and c can be single or multiple.

Fig. 1 is a schematic flowchart of a log processing method 100 provided in this embodiment, where the method 100 may be executed by a server, for example, the cloud server described above, but this is not limited in this embodiment.

As shown in fig. 1, the method 100 may include the steps of:

s101, generating logs in different log formats for the application service.

When the application service runs on the server, the server can generate different types of logs based on the container technology, each log type corresponds to one log format, and the server can obtain the logs in different log formats.

It should be noted that the log format may refer to different formats of the log files, that is, different file formats of different types of log storage, for example, pdf format, word format, text format, and the like, and may also refer to different storage formats of the content in the log files, for example, different spaces between fields.

And S102, converting the logs in different log formats into logs in a preset format.

The preset format is a default format of the server, and may be any format, which is not specifically limited in this embodiment of the present application. The preset format may also be referred to as a unified format, which is not limited in the embodiments of the present application.

When the server runs the application service by using the container technology, a log in a preset format can be generated.

For example, the server may establish acquisition configuration components of different technical systems, and may call, for logs of different log formats, a corresponding acquisition configuration component to generate a log of a preset format to facilitate subsequent data acquisition.

S103, determining a processing strategy of the log according to the log type of the log in the preset format and a first corresponding relation, wherein the first corresponding relation comprises a plurality of groups of corresponding relations, each group of corresponding relations in the plurality of groups of corresponding relations is used for indicating the corresponding relation between the log type and the processing strategy, the processing strategy comprises an acquisition strategy and a sending strategy, the acquisition strategy is used for acquiring the log in the preset format, and the sending strategy is used for sending the acquired information for viewing.

And the log types are used for distinguishing different types of logs.

Optionally, the log type may include at least one of: transaction type logs, service logs, monitoring logs, system logs, or buried point analysis logs, but the embodiments of the present application are not limited thereto.

And the server determines a processing strategy corresponding to the log type of the log in the preset format in the first corresponding relation, wherein the processing strategy comprises an acquisition strategy and a sending strategy. The collection policy and the sending policy are only named examples, and the embodiment of the present application does not limit this.

Illustratively, if the log type of the log with the preset format is a monitoring log, a transaction log and the like, which have higher requirements on acquisition timeliness, the acquisition frequency in the acquisition policy is higher, and the transmission frequency in the transmission policy is higher. If the log type of the log in the preset log format is a log with low requirement on acquisition timeliness, such as a buried point analysis log, the acquisition frequency in the acquisition strategy is low, and the transmission frequency in the transmission strategy is low.

Optionally, after the server collects the information in the log with the preset format according to the collection policy, the collected data may be stored in a storage structure, for example, a storage structure such as a database, kafka, hdfs, or the like.

Optionally, the processing policy further includes an information filtering policy and a monitoring policy, the information filtering policy is used to filter information in the log, and the monitoring policy is used to monitor the acquisition policy and the sending policy to adjust the acquisition policy and the sending policy.

The information filtering strategy is used for filtering the information in the log with the preset format.

For example, if the log type of the log in the preset format is a monitoring log, the information filtering policy may be to only reserve fields to be monitored, and meanwhile, information such as acquisition time may be added for supplementation. If the log type of the log is a buried point analysis log, the information filtering policy of the log may be to only reserve the required buried point information. If the log type of the log is a transaction log, the information filtering policy may be to only retain transaction information.

The monitoring strategy is used for monitoring the acquisition strategy and the sending strategy so as to adjust the acquisition strategy and the sending strategy.

For example, the monitoring log and the transaction log have high real-time requirements, and the monitoring policy may be: when the transmission frequency of the monitoring log or the transaction log is high, the transmission of the system log and the buried point analysis log can be reduced or suspended.

And S104, processing the log in the preset format according to the processing strategy.

The server can collect logs in a preset format according to the collection strategy, and can send the collected information for viewing according to the sending strategy. The collecting the log in the preset format may include collecting information such as text data and stream data of the log in the preset format, but the embodiment of the present disclosure is not limited thereto.

The log processing method provided by the embodiment of the application can automatically convert logs in different log formats into logs in a preset format for application service, reduces the step of selecting an acquisition tool according to the format of the logs, determines the processing strategy of the logs in the preset format according to the log type and the first corresponding relation of the logs in the preset format, reduces the step of manually determining the processing strategy, reduces the workload of a user, improves the automation level, and is favorable for improving the log processing efficiency.

As an optional embodiment, before determining the processing policy of the log in the preset format according to the log type of the log in the preset format and the first corresponding relationship in S103, the method 100 may further include: classifying the logs in the preset format according to the log types of the logs in the preset format to obtain at least one category of logs; s103, determining a processing strategy of the log in the preset format according to the log type of the log in the preset format and the first corresponding relation, wherein the processing strategy comprises the following steps: determining a processing strategy of the log of each category in the logs of at least one category according to the log type corresponding to the log of at least one category and the first corresponding relation; s104, processing the log in the preset format according to the processing strategy, wherein the processing comprises the following steps: and processing the logs of each category according to the processing strategy of the logs of each category.

The server can classify the logs in the preset format according to the log types of the logs in the preset format to obtain at least one category of logs. The number of logs is plural and the server can attribute it to at least one category. The server may determine the log type of the log according to a field in the log in a preset format, but the embodiment of the present application is not limited thereto.

Illustratively, the log types may include transaction class logs, traffic logs, monitoring logs, and buried point analysis logs. The number of the logs in the preset format is multiple, the log types of the multiple logs comprise a service log, a monitoring log and a buried point analysis log, and the server can divide the multiple logs into 3 types to obtain 3 types of logs.

According to the log processing method provided by the embodiment of the application, logs in a preset format are classified, then different processing strategies are used for the logs in different types, the same processing strategy is used for the logs in the same type, the processing efficiency of each log does not need to be judged one by one, and the log processing efficiency can be further improved.

As an optional embodiment, the method 100 may further include: judging whether the log type of the log with the preset format is a preset type or not; if the log type of the first log in the logs with the preset format is the preset type, determining the processing strategy of the first log as a high-priority processing strategy.

The preset type may also be referred to as a preset log type, which is not limited in this embodiment of the present application. The preset type may be one log type or multiple log types, which is not limited in the embodiment of the present application.

The preset type may be a monitoring log and/or a transaction log, for example.

The processing strategy of the logs of the preset type has high priority, and if the log type of a first log in the logs is the preset type, the first log is collected and sent preferentially.

The log processing method provided by the embodiment of the application has the advantages that the high priority is set for the preset type of logs, important logs can be processed preferentially, the user requirements can be met, and the flexibility of log processing is improved.

Optionally, the method 100 may further include: monitoring the processing condition of the first log to obtain monitoring data; and adjusting the processing strategy of the first log according to the monitoring data.

The server can also monitor the processing condition of the first log so as to adjust the processing strategy in real time.

For example, the server may obtain monitoring data according to the processing condition of the first log, where the monitoring data may include: the method comprises the steps of collecting data flow, caching, sending data flow, network conditions and other data, and if the caching is detected to be 1 million and the utilization rate is 80%, the data flow can be reduced, and the data sending flow can be increased, so that normal operation of log processing is guaranteed.

According to the log processing method provided by the embodiment of the application, the processing condition of the first log which is processed preferentially can be monitored in real time, so that the processing strategy is optimized, and the processing strategy can be more suitable for application scenes.

Based on the log processing method, the embodiment of the present application provides a specific implementation manner.

Illustratively, FIG. 2 shows a schematic block diagram of a log collection tool. The server can implement the log processing method provided by the embodiment of the application through the log collecting tool.

As shown in FIG. 2, the log collection tool can include an application execution component, a log collection component, a log transmission component, and a storage component. Wherein the application execution assembly comprises an acquisition probe. The log collection component comprises a configuration translator, a collection monitor, a controller, a data collector, a data filter and a data collector. The log sending component comprises a data sender and a data monitor. The application service shown in fig. 2 is used to represent the source of the log.

Wherein the acquisition probe can be used to:

1) logs of different log formats generated by different technology stacks are read and transmitted to the configuration translator. Wherein different technology stacks are introduced when container technology is utilized.

2) And if the log type of the log is a preset type, transmitting the log to an acquisition monitor.

The translator is configured to:

1) establishing acquisition configuration components for different technical systems for being called by a server, thereby generating a log with a preset format and facilitating subsequent data acquisition;

2) and receiving the logs from the acquisition probe, classifying the logs according to the log types of the logs to obtain logs of at least one category, and sending the logs of the at least one category to the controller.

The acquisition monitor is used for: the method comprises the steps of receiving a preset type log from an acquisition probe, monitoring the processing condition of the log to obtain monitoring data, sending the monitoring data to a controller, and transmitting the monitoring data to a data storage.

The controller is configured to:

1) the method comprises the steps of receiving at least one category of logs from a configuration translator, determining a processing strategy of the at least one category of logs according to a log type corresponding to the at least one category of logs, wherein the processing strategy can comprise a collection strategy, an information filtering strategy, a sending strategy and a monitoring strategy, sending the sending strategy to a data collector, sending the information filtering strategy to a data filter, sending the sending strategy to a data sender and sending the monitoring strategy to a data monitor.

2) And receiving monitoring data from the acquisition monitor, and adjusting the processing strategy in real time according to the monitoring data.

3) And receiving monitoring data of the data monitor, and adjusting the processing strategy in real time according to the monitoring data.

The data collector is used for:

1) the method comprises the steps of receiving an acquisition strategy from a controller, acquiring logs generated by application services according to the acquisition strategy, and sending acquired data (which can be simply referred to as acquired data) to a data filter.

2) The acquisition strategy can be dynamically adjusted according to the real-time data of the acquisition monitor.

3) And sending the collected data to a data memory.

The data filter is to: the method comprises the steps of receiving an information filtering strategy from a controller, receiving collected data from a data collector, carrying out corresponding screening and filtering on the collected data according to the information filtering strategy, and sending the filtered data (which can be referred to as filtering data for short) to a data sender.

The data transmitter is configured to: and receiving a sending strategy from the controller, receiving filtered data from the data filter, and determining the sending data according to the sending strategy.

The data monitor is to: and receiving the monitoring strategy from the controller, respectively receiving the collected data, the filtered data and the sent data from the data collector, the data filter and the data sender, monitoring the collected data, the filtered data and the sent data, and sending the monitored data to the data memory and the controller.

The data storage is to:

1) and receiving and storing the collected data from the data collector.

2) Monitoring data from the acquisition monitor is received.

3) Monitoring data from a data monitor is received.

Based on the log collection tool provided above, the embodiment of the present application further provides a log processing method.

Illustratively, FIG. 3 shows a schematic flow diagram of a log processing method 300. As shown in fig. 3, the method 300 includes the steps of:

s301, starting application service.

Application service launch may be understood as the application service beginning to run.

S302, the acquisition probe generates logs in different log formats for the application service.

This step can refer to S101 described above, and is not described here.

S303, the acquisition probe sends logs in different log formats to the configuration translator, and correspondingly, the configuration translator receives the logs in different log formats.

S304, the configuration translator converts the logs in different log formats into logs in a preset format.

The configuration translator can generate logs in a preset format for the application service through acquisition configuration components of different technical systems.

Optionally, the configuration translator may further determine whether the log type of the log in the preset format is a preset type, and if the log type of the log in the preset format is the preset type, may transmit the log in the preset format to the acquisition monitor. After receiving the log in the preset format, the acquisition monitor can monitor the processing condition of the log in the preset format to obtain monitoring data, and sends the monitoring data to the controller to adjust the processing strategy of the log in the preset format in real time.

Optionally, the monitoring data of the collecting monitor can also be stored in the data storage.

S305, the configuration translator classifies the logs in the preset format according to the log types of the logs in the preset format to obtain at least one category of logs.

S306, the configuration translator sends the logs of at least one category to the controller, and correspondingly, the controller receives the logs of the at least one category.

S307, the controller determines a processing strategy of the logs of the at least one category according to the log type corresponding to the logs of the at least one category, wherein the processing strategy can comprise an acquisition strategy, an information filtering strategy, a sending strategy and a monitoring strategy.

The controller can also send the sending strategy to the data collector, correspondingly, the data collector receives the sending strategy, collects the logs in the preset format according to the collecting strategy, sends the collected data to the data filter, and correspondingly, the data filter receives the collected data.

The controller can also send the information filtering strategy to the data filter, correspondingly, the data filter receives the information filtering strategy, correspondingly filters the collected data according to the information filtering strategy, and sends the filtered data to the data sender, and correspondingly, the data sender receives the filtered data.

The controller may also transmit a transmission policy to the data transmitter, and correspondingly, the data transmitter receives the transmission policy and determines to transmit data according to the transmission policy.

The controller can also send the monitoring strategy to the data monitor, acquire the collected data, the filtered data and the sent data of the data collector, the data filter and the data sender, and monitor the data to adjust the processing strategy in real time.

Optionally, the monitoring data of the data monitor and the collected data of the data collector can also be stored in the data storage.

According to the log processing method provided by the embodiment of the application, logs in different log formats are automatically converted into logs in a preset format for application services through the log collecting tool, the logs in the preset format are classified according to the log types of the logs in the preset format, a processing strategy is determined for each category of logs, the generation, collection and sending of the logs are automatically realized, and the log processing efficiency is improved.

The log processing method provided by the embodiment of the present application is described in detail above with reference to fig. 1 to 3, and the log processing apparatus provided by the embodiment of the present application is described in detail below with reference to fig. 4 and 5.

Fig. 4 illustrates a log processing apparatus 400 according to an embodiment of the present application. The apparatus 400 comprises: a generation module 410 and a processing module 420. The generation module 410 is configured to: serving the application with logs of different log formats; the processing module 420 is configured to: converting the logs in different log formats into logs in a preset format; determining a processing strategy of the log according to the log type of the log in a preset format and a first corresponding relation, wherein the first corresponding relation comprises a plurality of groups of corresponding relations, each group of corresponding relation in the plurality of groups of corresponding relations is used for indicating the corresponding relation between the log type and the processing strategy, the processing strategy comprises an acquisition strategy and a sending strategy, the acquisition strategy is used for acquiring the log in the preset format, and the sending strategy is used for sending the acquired information for viewing; and processing the log in the preset format according to the processing strategy.

Optionally, the processing module 420 is further configured to: classifying the logs in the preset format according to the log types of the logs in the preset format to obtain at least one category of logs; determining a processing strategy of the log of each category in the logs of at least one category according to the log type corresponding to the log of at least one category and the first corresponding relation; and processing the logs of each category according to the processing strategy of the logs of each category.

Optionally, the processing module 420 is further configured to: if the log type of the first log in the logs with the preset format is the preset type, determining the processing strategy of the first log as a high-priority processing strategy.

Optionally, the processing module 420 is further configured to: monitoring the processing condition of the first log to obtain monitoring data; and adjusting the processing strategy of the first log according to the monitoring data.

Optionally, the log type of the log in the preset format includes at least one of: transaction type logs, service logs, monitoring logs, system logs, or buried point analysis logs.

It should be appreciated that the apparatus 400 herein is embodied in the form of functional modules. The term module, as used herein, may refer to an Application Specific Integrated Circuit (ASIC), an electronic circuit, a processor (e.g., a shared, dedicated, or group processor), and memory that execute one or more software or firmware programs, a combinational logic circuit, and/or other suitable components that support the described functionality. In an alternative example, as can be understood by those skilled in the art, the apparatus 400 may be specifically a server in the foregoing embodiment, or the functions of the server in the foregoing embodiment may be integrated in the apparatus 400, and the apparatus 400 may be configured to perform each process and/or step corresponding to the server in the foregoing method embodiment, and is not described herein again to avoid repetition.

The device 400 has functions of implementing corresponding steps executed by the server in the method 100; the above functions may be implemented by hardware, or may be implemented by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the functions described above.

Fig. 5 illustrates a log processing apparatus 500 according to an embodiment of the present application. The apparatus 500 comprises: a processor 510, a communication interface 520, and a memory 530. Wherein the processor 510, the communication interface 520 and the memory 530 are in communication with each other via an internal connection path, the memory 530 is configured to store instructions, and the processor 510 is configured to execute the instructions stored in the memory 530 to control the communication interface to transmit and/or receive signals.

It should be understood that the apparatus 500 may be configured to execute each step and/or flow corresponding to the server in the method embodiments described above. Alternatively, the memory 530 may include a read-only memory and a random access memory, and provides instructions and data to the processor 510. A portion of memory 530 may also include non-volatile random access memory. For example, memory 530 may also store device type information. The processor 510 may be configured to execute the instructions stored in the memory 530, and when the processor 510 executes the instructions stored in the memory 530, the processor 510 is configured to perform the steps and/or processes of the method embodiments corresponding to the server described above.

It should be understood that, in the embodiment of the present application, the processor 510 of the apparatus 500 may be a Central Processing Unit (CPU), and the processor 510 may also be other general processors, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or instructions in the form of software. The steps of a method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware processor, or may be implemented by a combination of hardware and software elements in a processor. The software elements may be located in ram, flash, rom, prom, or eprom, registers, etc. as is well known in the art. The storage medium is located in a memory, and a processor executes instructions in the memory and combines hardware thereof to perform the steps of the above-described method. To avoid repetition, it is not described in detail here.

The present application provides a readable computer storage medium for storing a computer program for implementing the method corresponding to the server in the above embodiments.

The present application provides a computer program product comprising a computer program (which may also be referred to as code, or instructions) which, when run on a computer, can carry out the method corresponding to the server in the above embodiments.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of a unit is merely a logical division, and an actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the methods according to the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily think of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A log processing method, comprising:

generating logs in different log formats for the application service;

converting the logs in different log formats into logs in a preset format;

determining a processing strategy of the log in the preset format according to the log type of the log in the preset format and a first corresponding relation, wherein the first corresponding relation comprises a plurality of groups of corresponding relations, each group of corresponding relation in the plurality of groups of corresponding relations is used for indicating the corresponding relation between the log type and the processing strategy, the processing strategy comprises an acquisition strategy and a sending strategy, the acquisition strategy is used for acquiring the log in the preset format, and the sending strategy is used for sending the acquired information for viewing;

and processing the log with the preset format according to the processing strategy.

2. The method according to claim 1, wherein before determining the processing policy of the log in the preset format according to the log type of the log in the preset format and the first corresponding relationship, the method further comprises:

classifying the logs in the preset format according to the log types of the logs in the preset format to obtain at least one category of logs;

the determining the processing strategy of the log in the preset format according to the log type of the log in the preset format and the first corresponding relation comprises the following steps:

determining a processing strategy of the log of each category in the logs of the at least one category according to the log type corresponding to the log of the at least one category and the first corresponding relation;

the processing the log with the preset format according to the processing strategy comprises the following steps:

and processing the logs of each category according to the processing strategy of the logs of each category.

3. The method of claim 1, further comprising:

and if the log type of the first log in the logs with the preset format is a preset type, determining the processing strategy of the first log as a high-priority processing strategy.

4. The method of claim 3, further comprising:

monitoring the processing condition of the first log to obtain monitoring data;

and adjusting the processing strategy of the first log according to the monitoring data.

5. The method of claim 1, wherein the processing policy further comprises an information filtering policy and a monitoring policy, the information filtering policy is used for filtering information in the log, and the monitoring policy is used for monitoring the collection policy and the sending policy to adjust the collection policy and the sending policy.

6. The method according to any one of claims 1 to 5, wherein the log type of the log of the preset format comprises at least one of:

transaction type logs, service logs, monitoring logs, system logs, or buried point analysis logs.

7. A log processing apparatus, comprising:

the generating module is used for generating logs in different log formats for the application service;

the processing module is used for converting the logs in different log formats into logs in a preset format; determining a processing strategy of the log in the preset format according to the log type of the log in the preset format and a first corresponding relation, wherein the first corresponding relation comprises a plurality of groups of corresponding relations, each group of corresponding relation in the plurality of groups of corresponding relations is used for indicating the corresponding relation between the log type and the processing strategy, the processing strategy comprises an acquisition strategy and a sending strategy, the acquisition strategy is used for acquiring the log in the preset format, and the sending strategy is used for sending the acquired information for viewing; and the number of the first and second groups,

and processing the log in the preset format according to the processing strategy.

8. The apparatus of claim 7, wherein the processing module is further configured to:

and processing the log of each category according to the processing strategy of the log of each category.

9. The apparatus of claim 7, wherein the processing module is further configured to:

10. The apparatus of claim 9, wherein the processing module is further configured to:

monitoring the processing condition of the first log to obtain monitoring data;

11. The apparatus of claim 7, wherein the processing policy further comprises an information filtering policy and a monitoring policy, the information filtering policy is configured to filter information in the log, and the monitoring policy is configured to monitor the collection policy and the sending policy to adjust the collection policy and the sending policy.

12. The apparatus according to any one of claims 7 to 11, wherein the log type of the log of the preset format comprises at least one of:

13. A log processing apparatus, comprising: a processor coupled with a memory for storing a computer program that, when invoked by the processor, causes the apparatus to perform the method of any of claims 1 to 6.

14. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a computer program comprising instructions for implementing the method according to any one of claims 1 to 6.

15. A computer program product comprising computer program code which, when run on a computer, causes the computer to carry out the method according to any one of claims 1 to 6.