CN102497431B - Memory application method and system for caching application data of transmission control protocol (TCP) connection - Google Patents

Abstract

The invention provides a method for caching the application data of transmission control protocol (TCP) connection. The method is characterized in that: the TCP connection applies for a buffer block with a fixed length from a static buffer pool under the condition of small application load, and dynamically applies for a buffer block with a fixed length from an operating system under the condition of high application load. Compared with the prior art, the method has the advantages that: upper-layer application is well supported to temporarily store a load for content analysis; when the upper-layer application has a small memory load, resources can be quickly acquired from the static buffer pool; and when the upper-layer application has a heavy memory load, data can be properly buffered to avoid packet loss.

Description

A memory application method and system for TCP connection caching application data

technical field

The invention belongs to the field of network security, and in particular relates to a memory application method and system for TCP connection cache application data.

Background technique

With the rapid development of the Internet, the Internet has brought convenience to people, but at the same time it has brought many problems. Such as: pornography, anti-government public opinion, etc. can be transmitted through the network. Therefore, the importance of network monitoring is becoming more and more significant. Most of the current network adopts the four-layer model of TCP/IP. To monitor the data content in the application layer, the application load content of the data packet must be checked. Under the TCP/IP model, it is only necessary to analyze the load at the transport layer. Based on the application of TCP connection, the data of each TCP connection can be checked to determine whether its content is illegal.

Patent No. CN200580031571.0 (Caching Content and State Data at Network Elements) discloses a method for caching content and state data at network elements. In one embodiment, data is intercepted at a network element. An application layer message specifying a request for the specified data to be applied to the server is determined from the data packet. A first portion of the specified data contained in the cache of the network element is determined. A message is sent to the server application requesting a second portion of the data not contained in the cache. A first response containing the second part but not the first part is received. A second response including the first and second parts is sent to the client application. In one embodiment, data packets are intercepted at a network element. An application-layer message that determines state information for a specified session or database connection from a data packet. State information is cached at network elements.

Patent No. CN200680012181.3 (distributed data management system and its method for dynamically subscribing data) discloses a distributed data management system, including: an application module (1) and a data manager (2); the application module (1 ) is provided with a data access module (11) and a data cache (12); the data manager (2) is provided with a subscription management module (21), a subscription list module (22), a notification module (23) and a data memory (24); in addition, a dynamic subscription management module (14) and a data recording module (15) are also provided in the application module (1); a data release module (25) is also provided in the data manager (2) ), the data release module (25) is connected to the data store (24); the dynamic subscription management module (14) is respectively connected to the data recording module (15), the data cache (12) and the The data access module (11) is connected to communicate with the subscription management module (21), the notification module (23) and the data publishing module (25); the invention also includes a method for dynamically subscribing to data. By adopting the invention, the amount of data transmitted by the network and processed by the system can be effectively reduced, the burden on the network can be reduced, and the working performance of the system can be improved.

In the above tcp offloading system, software and hardware are not configured or only a small amount of buffers are configured for caching application data.

The disadvantage of the above technique is: in the tcp offloading system, the hardware is not configured or only a small amount of buffer is configured for caching application data. It does not check the load content of the TCP connection, and does not support temporary caching of some data for the upper-layer application. Therefore, it cannot cooperate with the content analysis of the upper-layer application well, and it can only lose packets when the upper-layer application is busy.

Contents of the invention

The present invention overcomes the deficiencies of the prior art, provides an applied cache allocation mechanism, and can statically allocate a certain capacity of memory for each connection. When the memory is not enough, the dynamic and static combination of memory resources is obtained from the OS through dynamic allocation, which saves resources and can be used as soon as possible. meet application needs.

The present invention provides a kind of memory application method of TCP connection cache application data, and it comprises the following steps:

(1) Initialize, according to the application scale, apply for several nodes to form a static pool according to various scales (such as three scales, 5k, 1.5k, 0.5k) for the stream cache node, and go to step (2);

(2) flow node application to step (3); flow node release to step (7);

(3) Apply for an idle node from the static pool, if the application is successful, proceed to step (5), otherwise proceed to step (4);

(4) Apply for a dynamic streaming cache node (the size is the smallest one that can meet the requirements in the static state) from the operating system, if the application is successful, then enter step (5), otherwise enter step (6);

(5) return node head pointer, go to step (11);

(6) return null pointer, go to step (11);

(7) If the stream cache node has a dynamic application mark, go to step (8), otherwise go to step (9);

(8) In the static pool, if the number of flow cache nodes of the same scale is less than the set threshold (eg: 1k), go to step

(9), otherwise go to step (10);

(9) The flow cache node is put into the static pool, and then step (11) is turned on;

(10) The stream cache node is returned to the operating system, and then step (11);

(11) END.

In the memory application method for TCP connection cache application data provided by the present invention, in step (3), the TCP connection applies for a buffer block with a fixed length len from the free buffer list in the static buffer pool.

In the memory application method for TCP connection cache application data provided by the present invention, in step (4), the TCP connection dynamically applies for a buffer block with a fixed length len from the operating system.

The memory application method of the TCP connection cache application data provided by the present invention, in step (7)～(10), according to the mark of the released node and the number of nodes of the same scale in the static pool (the preset threshold value of the same scale as in the static pool Compare) to determine whether the node is reclaimed by the static pool or by the operating system.

In the memory application method for TCP connection cache application data provided by the present invention, when the TCP connection is closed or eliminated, the modules (7)-(10) of node recovery are also used for processing.

The present invention also provides a system for memory application of TCP connection cache application data, which includes the following modules:

(1) The initialization module, according to the application scale, applies for several nodes to form a static pool according to various scales for the stream cache node;

(2) Apply for the static buffer block module, and the flow node applies for the transfer module (3); the flow node releases the transfer module (7);

(3) Apply for idle nodes from the static pool, if the application is successful, enter module (5), otherwise enter module (4);

(4) Apply for a dynamic buffer module, apply for a dynamic stream cache node from the operating system, if the application is successful, then enter module (5), otherwise enter module (6);

(5) return node head pointer, turn module (11);

(6) return null pointer, turn module (11);

(7) The stream cache node has a dynamic application flag and transfers to the module (8), otherwise it transfers to the module (9);

(8) In the static pool, if the number of flow cache nodes of the same scale is less than the set threshold, transfer to module (9), otherwise transfer to module (10);

(9) The flow cache node is put into the static pool, and then the module (11) is transferred;

(10) The stream cache node is returned to the operating system, and then transferred to the module (11);

(11) end;

Among them, the scales described in the initialization module include three types, namely: 5k, 1.5k, and 0.5k; the node size described in the application dynamic buffer module is the smallest one that can meet the requirements in the static state.

The present invention also provides a memory application system for TCP connection caching application data. In the module (2), the TCP connection applies to obtain a buffer block with a fixed length len from a free buffer list in a static buffer pool.

The present invention also provides a memory application system for TCP connection caching application data. In the module (4), the TCP connection applies for a buffer block with a fixed length len from the free buffer list in the dynamic buffer pool.

The present invention also provides the memory release system of the TCP connection cache application data, in the module (7, 8, 9, 10), according to the sign of the flow buffer node (dynamic application or static application) and the number of flow buffer nodes in the static buffer pool, Determines whether buffer nodes are reclaimed to the static pool or to the operating system.

In the memory application system for TCP connection caching application data provided by the present invention, when the TCP connection is closed or eliminated, the modules (7)-(10) of node recovery are also used for processing.

Compared with the prior art, the beneficial effect of the present invention is that: it well supports the temporary storage load of the upper layer application for content analysis, and can also perform proper buffering to avoid packet loss when the CPU load of the upper layer application is heavy; Combined with dynamic application, both application speed and application efficiency are considered. When the system stream cache node consumes less, it is directly obtained from the static pool to achieve fast application. When the system stream cache node consumes a lot, it is obtained from the operating system, which effectively uses the operating system resources. Combining static release and dynamic release, the stream cache nodes applied from the static pool are released to the static pool, and the dynamic nodes applied from the operating system are released to the static pool or the operating system according to the static nodes consumed by the system. It can be divided into the following points: 1. Avoid frequent application and release of stream cache nodes from the operating system, and a certain number of caches are directly applied and released from the stream cache node pool that is preferentially applied for; 2. When the stream cache nodes in the static pool When it is not enough, you can apply from the operating system to meet the application requirements; 3. When the stream cache node is released, according to the flag of the stream cache node (obtained from the static cache or dynamically obtained by the operating system) and the system's idle stream cache in the static pool According to the situation of the node, it is judged whether it is released to the static pool or the operating system.

Description of drawings

Fig. 1 is a schematic flow chart of the present invention.

Detailed ways

Referring to the schematic flow sheet of the present invention of Fig. 1, method of the present invention is carried out as follows:

1. Apply for a buffer block of fixed length len (the len mentioned later is len of fixed length) from the free buffer list in the static buffer pool for the TCP connection.

2. If step 1 fails, the static memory is not enough, then dynamically apply for the buffer of len, if the application fails, it means that the system resources are exhausted, return empty, if the dynamic application is successful, then connect the information nodes of this buffer to the dynamic linked list , record the dynamic flag in the information node, if step 1 is successful, connect the buffer block to the static linked list, and record the dynamic flag in the information node;

3. When the TCP connection is closed or eliminated, return the buffer to the system or put it back into the static free list according to the flag of the information node.

The present invention first statically allocates a piece of cache as a cache pool to satisfy the cache of TCP flow under normal traffic, dynamically allocates the cache when the traffic is large, and returns the dynamically allocated cache to the system according to a certain strategy after the traffic returns to the normal level. The data uploaded to the application is buffered and uploaded to the application as required. Therefore, the problem of providing a memory allocation mechanism for caching application layer data for tcp connections is solved.

The above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: the specific implementation of the present invention can still be carried out. Any modification or equivalent replacement without departing from the spirit and scope of the present invention shall fall within the scope of the claims of the present invention.

Claims (8)

1. a method for TCP Connection Cache application data, it comprises the steps:

(1) initialization,, goes to step (2) for stream cache node forms static pond by the several nodes of multiple size application according to application scale;

(2) application of stream cache node goes to step (3); Stream cache node discharges and goes to step (7);

(3) from static pond, apply for idle node, if applied for successfully, enter step (5), otherwise enter step (4);

(4), from operating system application dynamic flow cache node, if applied for successfully, enter step (5), otherwise enter step (6);

(5) return node head pointer, goes to step (11);

(6) return to null pointer, go to step (11);

(7) stream cache node has dynamic application mark to go to step (8), otherwise goes to step (9);

(8) in static pond, be less than setting threshold with the stream cache node size phase homogeneous turbulence cache node number of current release, go to step (9), otherwise go to step (10);

(9) stream cache node is put into static pond, goes to step (11);

(10) stream cache node returns to operating system, goes to step (11);

(11) finish;

Wherein, the size that flows cache node described in step (1) comprises three kinds, is respectively: 5KB, 1.5KB, 0.5KB; The size of dynamic flow cache node described in step (4) is the minimum one stream cache node size that can meet application dynamic flow cache node demand in static pond;

In step (7)～(10) according in the stream dynamic application mark of cache node and static pond with the stream cache node size phase homogeneous turbulence cache node number of current release, determine that stream cache node is reclaimed by static pond or operating system reclaims.

2. method according to claim 1, is characterized in that, in step (3), TCP connects the buffer blocks of applying for obtaining regular length len in the freebuf chained list from static pond.

3. method according to claim 1 and 2, is characterized in that, in step (4), TCP connects the buffer blocks of dynamically applying for obtaining regular length len from operating system.

4. method according to claim 1, is characterized in that, described TCP connection closed or when superseded, process step (7)～(10) that also adopt node to reclaim.

5. a system for the internal memory application of TCP Connection Cache application data, it comprises as lower module (1)～(11):

Module (1): initialization module, forms static pond for flowing cache node by the several nodes of multiple size application according to application scale;

Module (2): stream cache node application revolving die piece (3); Stream cache node discharges revolving die piece (7);

Module (3): apply for idle node from static pond, if applied for successfully, enter module (5), otherwise enter module (4);

Module (4): from operating system application dynamic flow cache node, if applied for successfully, enter module (5), otherwise enter module (6);

Module (5): return node head pointer, revolving die piece (11);

Module (6): return to null pointer, revolving die piece (11);

Module (7): stream cache node has dynamic application mark, revolving die piece (8), otherwise revolving die piece (9);

Module (8): in static pond, be less than setting threshold, revolving die piece (9), otherwise revolving die piece (10) with the stream cache node size phase homogeneous turbulence cache node number of current release;

Module (9): stream cache node is put into static pond, revolving die piece (11);

Module (10): stream cache node returns to operating system, revolving die piece (11);

Module (11): finish;

Wherein, the size that flows cache node described in initialization module comprises three kinds, is respectively: 5KB, 1.5KB, 0.5KB; The size of dynamic flow cache node described in module (4) is the minimum one stream cache node size that can meet application dynamic flow cache node demand in static pond;

In module (7)～(10) according in the stream dynamic application mark of cache node and static pond with the stream cache node size phase homogeneous turbulence cache node number of current release, determine that stream buffer joint reclaims to static pond or operating system.

6. system according to claim 5, is characterized in that, in module (3), TCP connects the buffer blocks of applying for obtaining regular length len in the freebuf chained list from static pond.

7. according to the system described in claim 5 or 6, it is characterized in that, in module (4), TCP connects the buffer blocks of dynamically applying for obtaining regular length len from operating system.

8. system according to claim 5, is characterized in that, described TCP connection closed or when superseded, (7)～(10) module that also adopts node to reclaim is processed.