CN119906757B - Method and system for optimizing eBPF monitoring services based on dynamic subscription mechanism - Google Patents

Abstract

The invention provides a method and a system for optimizing eBPF monitoring services based on a dynamic subscription mechanism, wherein the method comprises the steps of newly adding an SDK encapsulation interface function for a user to subscribe to a topic event, setting a subscription value for monitoring other user subscription/cancellation events based on a GRPC framework comprising a GRPC server and a client component, newly adding a eBPF map object B for storing the subscription value for dynamic subscription state synchronization, optimizing a eBPF kernel program, setting execution logic triggered by the kernel hook function according to the subscription value, and dynamically updating eBPF map object B by a eBPF user state program through subscription value change to realize dynamic control. The invention reduces the system resource waste caused by unsubscribing topic, dynamically adjusts the processing logic of the kernel hook function, and ensures the flexibility and instantaneity of monitoring service.

Description

Method and system for optimizing eBPF monitoring service based on dynamic subscription mechanism

Technical Field

The present invention belongs to the field of computer information technology, and specifically relates to a method and system for optimizing the system activity monitoring service performance of eBPF based on a dynamic subscription mechanism.

Background Art

Currently, the eBPF-based system activity monitoring service on the market has been widely used in security-aware services. By subscribing to different activity information (topics), refined monitoring of system behavior can be achieved. However, this monitoring mechanism has obvious shortcomings in performance optimization.

In the prior art, the system activity monitoring service based on eBPF usually loads all monitoring logic when the system starts. Regardless of whether the topic is subscribed, all hook functions will execute the complete processing flow when triggered, including data parsing, filtering, formatting and other operations. This design lacks flexibility. When the security-aware service does not subscribe to any topic or only subscribes to a small number of topics, the unsubscribed topics will still trigger the eBPF program in the kernel and the user-mode processing flow, resulting in the processing burden of unsubscribed topics still existing, causing a waste of system resources and affecting the overall performance of the system. In particular, in high-frequency triggering scenarios, this redundant processing has a particularly significant impact on system performance.

Summary of the invention

The purpose of the present invention is to provide a method and system for optimizing eBPF monitoring services based on a dynamic subscription mechanism, reduce the waste of system resources caused by unsubscribed topics, dynamically adjust the processing logic of kernel hook functions, and ensure the flexibility and real-time performance of monitoring services.

In order to achieve the above object, the technical solution of the present invention is as follows:

A method for optimizing eBPF monitoring service based on a dynamic subscription mechanism, comprising:

S1. Add SDK encapsulation interface functions for users to subscribe to topic events. The SDK is based on the GRPC framework and includes GRPC server and client components.

S2. Set the subscription value used to monitor other users’ subscription/cancellation events, and add a new eBPFmap object B to save the subscription value for dynamic subscription state synchronization;

S3, optimize the eBPF kernel program, and set the execution logic after the kernel hook function is triggered according to the subscription value;

S4. The eBPF user-state program dynamically updates the eBPF map object B by subscribing to value changes to achieve dynamic control.

Furthermore, the interface function in step S1 includes subscribing to events, unsubscribing, publishing events, and reading events. The interface function is directly mapped to the corresponding operations in the GRPC framework for users to call seamlessly after connecting to the GRPC client.

Furthermore, the method for eBPF map object B to save subscription value in step S2 includes:

S201, after the eBPF user state program is started, it first subscribes itself as a system user, calls the interface function for reading events and waits to obtain event information subscribed by other users;

S202. When other users call the interface function for subscribing to events or unsubscribing, the GRPC server calls the interface function for publishing events to publish user information, subscription events, or types of unsubscriptions. At this time, the eBPF user-mode program receives the subscription value.

S203. After receiving the subscription value, the eBPF user-mode program writes a newly added eBPF map object B for the eBPF kernel program to read and use.

Furthermore, the optimization of the eBPF kernel program in step S3 includes:

S301, after the kernel hook function is triggered, the eBPF kernel program is executed, firstly reading the current subscription value from the eBPF map object B and determining whether the subscription value corresponding to the current kernel hook function is in the subscription state;

S302, if not subscribed, directly terminate the eBPF kernel program and return;

S303: If subscribed, continue with the subsequent data processing flow.

Further, step S4 includes:

When the user's subscription status changes, the eBPF user-state program obtains the changed subscription value through the GRPC server and dynamically updates the eBPF map object B.

On the other hand, the present invention also proposes a system for optimizing eBPF monitoring services based on a dynamic subscription mechanism, comprising:

SDK module: Added SDK encapsulation interface functions for users to subscribe to topic events. The SDK is based on the GRPC framework and includes GRPC server and client components.

Subscription value module: sets the subscription value used to monitor other users' subscription/cancellation events, and adds an eBPF map object B to save the subscription value for dynamic subscription status synchronization;

Optimization module: optimizes the eBPF kernel program and sets the execution logic after the kernel hook function is triggered according to the subscription value;

Dynamic control module: The eBPF user-state program dynamically updates the eBPF map object B by subscribing to value changes to achieve dynamic control.

Furthermore, the interface functions in the SDK module include subscribing to events, unsubscribing, publishing events, and reading events. The interface functions are directly mapped to corresponding operations in the GRPC framework for users to call seamlessly after connecting to the GRPC client.

Furthermore, the subscription value module includes:

Waiting unit: After the eBPF user-mode program is started, it first subscribes itself as a system user, calls the interface function for reading events, and waits to obtain event information subscribed by other users;

Receiving unit: When other users call the interface function of subscribing to events or unsubscribing, the GRPC server calls the interface function of publishing events to publish user information, subscription events, or unsubscribe types. At this time, the eBPF user-mode program receives the subscription value.

Writing unit: After receiving the subscription value, the eBPF user-mode program writes the newly added eBPF map object B for the eBPF kernel program to read and use.

Furthermore, the optimization module includes:

Reading unit: After the kernel hook function is triggered, the eBPF kernel program is executed. First, the current subscription value is read from the eBPF map object B and whether the subscription value corresponding to the current kernel hook function is in the subscription state;

Return unit: If not subscribed, the eBPF kernel program ends directly and returns;

Continue unit: If subscribed, continue the subsequent data processing flow.

Furthermore, the dynamic control module includes:

When the user's subscription status changes, the eBPF user-state program obtains the changed subscription value through the GRPC server and dynamically updates the eBPF map object B.

Compared with the prior art, the present invention has the following beneficial effects:

1. The present invention reduces redundant processing: the kernel and user-mode processing flows corresponding to unsubscribed topics are immediately interrupted, thus avoiding unnecessary CPU and memory consumption.

2. The present invention has dynamic adjustment flexibility: when the subscription status changes, the kernel behavior can be adjusted instantly without restarting the service or manual intervention.

3. The performance of the present invention is significantly optimized: in high-frequency triggering scenarios, the utilization rate of system resources is significantly improved, especially for large-scale monitoring scenarios.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic diagram of a flow chart of Embodiment 1 of the present invention.

FIG. 2 is an overall framework diagram of Embodiment 1 of the present invention.

FIG3 is a schematic diagram of the dynamic control flow of Example 1 of the present invention.

DETAILED DESCRIPTION

It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

First, the professional terms involved in the embodiments are explained.

eBPF (Extended Berkeley Packet Filter): An efficient mechanism for executing user-defined programs in the kernel, used to monitor and debug system behavior.

GRPC: A high-performance, cross-language remote procedure call framework for communication between services in distributed systems.

Hook function: A specific trigger point in the operating system kernel that is used to intercept system events and perform custom processing.

Map object: A data structure in eBPF used to share data between the kernel and userland.

SDK (Software Development Kit): Software development kit used to develop applications.

The present invention is described in detail below in conjunction with specific embodiments and drawings.

Embodiment 1:

In this embodiment, a method for optimizing the performance of the eBPF system activity monitoring service based on a dynamic subscription mechanism is proposed, as shown in FIG1 , including:

Step 1: Add a new SDK for users to subscribe to events.

This step provides a new set of software development kits (SDKs) that allow users to interact with services built on the GRPC framework through simple interface functions. The SDK supports mainstream programming languages such as C/C++ and Python, and encapsulates the following four core interface functions:

1. Subscribe to events subscribe(): Users can subscribe to specific topic events through this interface to receive relevant notifications.

2. Unsubscribe(): Allows the user to stop receiving previously subscribed event notifications.

3. Publish events publish_events(): used to send new events to the system so that other subscribers can receive these events.

4. Read events read_events(): Users can query and obtain event records that have occurred through this interface.

The SDK implements a complete GRPC framework, including GRPC server and client components, to ensure efficient and reliable message delivery. As shown in the overall framework diagram of Figure 2, each interface function of the SDK is directly mapped to the corresponding operation in the GRPC framework, so that users can call these functions seamlessly after connecting to the GRPC client.

In addition, the SDK is designed to simplify the user's development process, allowing users to easily integrate event processing functions into their own applications without having to deeply understand the specific implementation details of GRPC.

Step 2: Add a new eBPF map object B for dynamic subscription state synchronization.

First, in this embodiment, each system call is assigned a subscription value, such as file reading 0x01, file writing 0x02, file creation 0x04, file deletion 0x08, process creation 0x10, network connection 0x100, etc. Subsequent operations will pass this subscription value to determine different processing.

Set a special subscription value: 0, which is used to monitor other users' subscription/cancellation events.

Next, the logic of the user-mode program saving the subscription value through the eBPF map object B includes:

(1) After the eBPF user-mode program is started, it first subscribes itself as a system user (subscription value: 0), calls the interface function for reading events (read_events) and waits to obtain event information subscribed by other users, as shown in the overall framework diagram of Figure 2.

(2) When other users call the subscription event (subscribe)/unsubscription (unsubscribe) interface function, the GRPC server will call the publish event (publish_events) interface function to publish user information and subscription/unsubscription type events. At this time, the eBPF user-mode program can receive the subscription value.

(3) After receiving the subscription value, the eBPF user-mode program writes the newly added eBPF map object B.

(4) eBPF map object B is a data structure map provided by eBPF. After being defined in the kernel program, it can share data between user-mode programs and kernel programs. It is the main data exchange method of eBPF. eBPF map object B is used to save the user's subscription value, which will be read and used in the kernel program.

Step 3: Optimize the execution logic after the kernel hook function is triggered.

(1) The eBPF kernel program uses kprobe, fentry, tracepoints, etc. to establish a connection with the hook function in the kernel. It can capture and execute custom logic when the hook function in the kernel is triggered.

(2) When the corresponding event occurs and the hook function is triggered, the eBPF kernel program will also be executed. First, it reads the current subscription value from the eBPF map object B to determine whether the subscription value corresponding to the current hook function is in the subscribed state.

(3) If the subscription value corresponding to the current hook function is not in the subscription state, the eBPF kernel program is terminated directly, and the system returns to the original processing flow, and continues to wait for the capture hook function to be triggered.

(4) If the subscription value corresponding to the current hook function is in the subscribed state, the subsequent data processing flow continues, including parsing, filtering, and sending to the eBPF user-mode program through map A to achieve interaction with the user-mode.

Step 4: Dynamic control process.

When the subscription status changes, the eBPF user-mode program receives the changed subscription value through the GRPC server and dynamically updates the eBPF map object B.

This mechanism ensures that data processing of unsubscribed topic events is interrupted immediately without restarting the service or reloading the eBPF program.

The complete dynamic control process is shown in Figure 3, including:

1. After the eBPF user-mode program is started, it first subscribes itself as a system user;

2. Call the event reading interface function (read_events) to wait for event information subscribed by other users;

3. User A calls the subscribe() interface function to subscribe to the file read (0x01) event, or calls the unsubscribe() interface function to cancel the subscription to the file read event;

4. The GRPC server sends (user A, 0x01) or (user A, 0x0) to the eBPF user program;

5. The eBPF user-mode program saves 0x01 or 0x0 to map B;

6. When the system call to read a file is triggered in the system, the vfs_read function in the kernel will be executed, and the corresponding fentry_vfs_read function in the eBPF kernel program will also be executed. The fentry_vfs_read function first reads the subscription value from map B; when the subscription value is 0x0 and the corresponding 0x01 is inconsistent, it will exit directly; when the subscription value is 0x01, it will continue to process, collect the uid, exe, file name and other information of the current process, and pass it to the user program through other means. If a file write event occurs, the hook function vfs_write is triggered, and the value 0x01 in map B is also read in the fentry_vfs_write function, which is inconsistent with the corresponding file write value (0x02), so it will also exit directly.

In the method of this embodiment, GRPC may also be replaced by other cross-process communication mechanisms (such as pipelines, semaphores, socket communication, etc.).

The method described in this embodiment solves the problem of how to reduce the waste of system resources caused by unsubscribed topics, solves the problem of how to dynamically adjust the processing logic of the kernel hook function to adapt to changes in subscription status, and solves the problem of how to ensure the flexibility and real-time performance of the monitoring service while reducing the system burden.

Embodiment 2:

This embodiment 2 proposes a system for optimizing eBPF monitoring services based on a dynamic subscription mechanism, including:

SDK module: Added SDK encapsulation interface functions for users to subscribe to topic events. The SDK is based on the GRPC framework and includes GRPC server and client components.

Subscription value module: sets the subscription value used to monitor other users' subscription/cancellation events, and adds an eBPF map object B to save the subscription value for dynamic subscription status synchronization;

Optimization module: optimizes the eBPF kernel program and sets the execution logic after the kernel hook function is triggered according to the subscription value;

Dynamic control module: The eBPF user-state program dynamically updates the eBPF map object B by subscribing to value changes to achieve dynamic control.

The interface functions in the SDK module include subscribing to events, unsubscribing, publishing events, and reading events. The interface functions are directly mapped to the corresponding operations in the GRPC framework for users to call seamlessly after connecting to the GRPC client.

Subscription value modules include:

Waiting unit: After the eBPF user-mode program is started, it first subscribes itself as a system user, calls the interface function for reading events, and waits to obtain event information subscribed by other users;

Receiving unit: When other users call the interface function of subscribing to events or unsubscribing, the GRPC server calls the interface function of publishing events to publish user information, subscription events, or unsubscribe types. At this time, the eBPF user-mode program receives the subscription value.

Writing unit: After receiving the subscription value, the eBPF user-mode program writes the newly added eBPF map object B for the eBPF kernel program to read and use.

The optimization modules include:

Reading unit: After the kernel hook function is triggered, the eBPF kernel program is executed. First, the current subscription value is read from the eBPF map object B and whether the subscription value corresponding to the current kernel hook function is in the subscription state;

Return unit: If not subscribed, the eBPF kernel program ends directly and returns;

Continue unit: If subscribed, continue the subsequent data processing flow.

The dynamic control module includes:

When the user's subscription status changes, the eBPF user-state program obtains the changed subscription value through the GRPC server and dynamically updates the eBPF map object B.

The system for optimizing the eBPF monitoring service based on the dynamic subscription mechanism proposed in this embodiment can implement the method for optimizing the eBPF monitoring service based on the dynamic subscription mechanism described in Embodiment 1, and has the same technical effect as Embodiment 1.

The above-mentioned embodiments are only preferred implementations of the present invention, and are only used to help understand the method and core ideas of the present application. The protection scope of the present invention is not limited to the above-mentioned embodiments. All technical solutions under the idea of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims (8)

1. A method for optimizing eBPF a monitoring service based on a dynamic subscription mechanism, comprising:

S1, a newly added SDK packaging interface function is used for a user to subscribe to a topic event, and the SDK is based on a GRPC framework and comprises a GRPC server and a client component;

S2, setting a subscription value for monitoring subscription/cancellation events of other users, and adding a eBPF map object B for storing the subscription value for dynamic subscription state synchronization;

s3, optimizing eBPF kernel programs, and setting execution logic triggered by a kernel hook function according to the subscription value;

S4, the eBPF user mode program dynamically updates eBPF map object B through subscription value change to realize dynamic control;

The step S2 comprises the following steps:

each system call is distributed with a subscription value, and the subsequent operation transmits the subscription value to determine different treatments, including file reading 0x01 and file writing 0x02;

the optimizing of eBPF kernel in step S3 includes:

s301, after a kernel hook function is triggered, eBPF kernel program execution is performed, firstly, a current subscription value is read from a eBPF map object B, and whether the subscription value corresponding to the current kernel hook function is in a subscription state or not is judged;

S302, if not subscribed, directly ending eBPF the kernel program and returning;

s303, if subscribed, continuing the subsequent data processing flow;

The step S4 includes:

When the system call of reading the file is triggered in the system, the vfs_read function in the kernel is executed, at the moment, the corresponding fentry _vfs_read function in the eBPF kernel program is also executed, the fentry _vfs_read function firstly reads the subscription value from the map B, when the subscription value is judged to be 0x0 and the corresponding 0x01 are inconsistent, the processing is directly exited, when the subscription value is judged to be 0x01, the processing is continued, the uid of the current process and the file name information of the exe file are collected and are transmitted to the user program in other modes, if a file writing event occurs, the hook function vfs_write is triggered, the value 0x01 in the map B is also read in the fentry _vfs_write function, and the corresponding file writing value 0x02 is inconsistent, and the processing is directly exited.

2. The method of claim 1, wherein the interface functions in step S1 include subscription events, unsubscribe, publish events, read events, and the interface functions are mapped directly to corresponding operations in the GRPC framework for seamless invocation by the user after connecting to the GRPC client.

3. The method for optimizing eBPF monitoring services based on dynamic subscription mechanism according to claim 1, wherein the method for storing the subscription value for eBPF map object B in step S2 includes:

S201, after a eBPF user mode program is started, subscribing the user as a system user, and calling an interface function for reading an event to wait for acquiring event information subscribed by other users;

S202, when other users call a subscription event or an interface function for unsubscribing, the GRPC server calls the interface function for publishing the event to publish the user information, the subscription event or the type for unsubscribing, and then eBPF user mode programs receive subscription values;

And S203, after the eBPF user mode program receives the subscription value, writing a newly added eBPF map object B for the eBPF kernel program to read.

4. The method for optimizing eBPF monitoring services based on dynamic subscription mechanisms as claimed in claim 1, wherein step S4 includes:

when the subscription state of the user changes, the eBPF user mode program obtains the changed subscription value through the GRPC server and dynamically updates eBPF map object B.

5. A system for optimizing eBPF monitoring services based on a dynamic subscription mechanism, comprising:

The SDK module is used for newly adding an SDK packaging interface function for a user to subscribe to the event of the topic, and the SDK is based on a GRPC framework and comprises a GRPC server and a client component;

The subscription value module is used for setting subscription values for monitoring subscription/cancellation events of other users, and adding a eBPF map object B for storing the subscription values for dynamic subscription state synchronization;

the optimizing module optimizes eBPF the kernel program, and sets execution logic triggered by the kernel hook function according to the subscription value;

the dynamic control module is used for dynamically updating eBPF map the object B through subscription value change by eBPF user mode programs to realize dynamic control;

The subscription value module comprises:

each system call is distributed with a subscription value, and the subsequent operation transmits the subscription value to determine different treatments, including file reading 0x01 and file writing 0x02;

The optimization module comprises:

the reading unit is used for reading the current subscription value from the eBPF map object B and judging whether the subscription value corresponding to the current core hook function is in a subscription state or not after the core hook function is triggered and eBPF core program execution is performed;

if not subscribed, directly ending eBPF the kernel program and returning;

If the data is subscribed, continuing the subsequent data processing flow;

The dynamic control module comprises:

When the system call of reading the file is triggered in the system, the vfs_read function in the kernel is executed, at the moment, the corresponding fentry _vfs_read function in the eBPF kernel program is also executed, the fentry _vfs_read function firstly reads the subscription value from the map B, when the subscription value is judged to be 0x0 and the corresponding 0x01 are inconsistent, the processing is directly exited, when the subscription value is judged to be 0x01, the processing is continued, the uid of the current process and the file name information of the exe file are collected and are transmitted to the user program in other modes, if a file writing event occurs, the hook function vfs_write is triggered, the value 0x01 in the map B is also read in the fentry _vfs_write function, and the corresponding file writing value 0x02 is inconsistent, and the processing is directly exited.

6. The system for optimizing eBPF monitoring services based on a dynamic subscription mechanism of claim 5 wherein the interface functions in the SDK module include subscribe events, unsubscribe, publish events, read events, and the interface functions map directly to corresponding operations in the GRPC framework for seamless invocation by a user after connecting to the GRPC client.

7. The system for optimizing eBPF monitoring services based on a dynamic subscription mechanism of claim 5, wherein the subscription value module comprises:

The waiting unit is used for subscribing the user as a system user after the eBPF user mode program is started, calling an interface function for reading the event and waiting for acquiring event information subscribed by other users;

When other users call the interface function of subscribing event or unsubscribing, the GRPC server calls the interface function of publishing event to publish the user information, subscribing event or unsubscribing type, and then eBPF user mode program receives the subscribing value;

And the writing unit is used for writing the newly added eBPF map object B for the eBPF kernel program to read after the eBPF user mode program receives the subscription value.

8. The system for optimizing eBPF monitoring services based on a dynamic subscription mechanism of claim 5, wherein the dynamic control module comprises:

when the subscription state of the user changes, the eBPF user mode program obtains the changed subscription value through the GRPC server and dynamically updates eBPF map object B.