CN115348208B - Flow control method and device, electronic equipment and storage medium - Google Patents

Abstract

The invention provides a flow control method, a flow control device, an electronic device and a computer storage medium, wherein the flow control method is applied to a first flow control component of a distributed file storage system, and comprises the following steps: after the first flow control component establishes connection with each gateway, receiving real-time flow of each catalogue reported by each gateway; determining the flow limit value of each gateway to each directory according to the real-time flow of each directory reported by each gateway and the preset flow limit value of each directory; and sending the flow limit value of each gateway to each directory to each gateway, so that each gateway controls the flow of each directory based on the flow limit value of each directory.

Description

A flow control method, device, electronic device and storage medium

Technical Field

The present invention relates to the field of computer application technology, and in particular to a flow control method, device, electronic equipment and computer storage medium.

Background technique

The International Data Corporation (IDC) predicts that the amount of global data is experiencing an explosion, growing from 33ZB in 2018 to 175ZB in 2025. By 2021, the amount of data stored in public clouds will exceed that in traditional data centers. The traditional data centers’ control over enterprise data since the 1960s will be broken by the cloud, and enterprises will enter the era of all-in-cloud. Cloud infrastructure (i.e., cloud storage) is divided into three types: file storage, object storage, and block storage, according to the different ways of data storage, organization, and data presentation format; it can be divided into symmetric and asymmetric types according to the different ways of metadata management. In the symmetric distributed file storage system, each node has an equal role, jointly manages and maintains metadata, and nodes perform information synchronization and mutex lock operations through a high-speed network. Figure 1a is a schematic diagram of the structure of a symmetric distributed file storage system in the related art. As shown in Figure 1a, the symmetric distributed file storage system consists of a storage cluster composed of a number of peer storage nodes, each storage node contains a number of storage buckets, and each storage bucket is generally connected to a local file system for storing client file data and metadata information. Each storage cluster belongs to the same namespace, which provides common file interface protocols for clients, such as the Network File System (NFS) protocol and the Common Internet File System (CIFS) protocol.

In the related art, for a symmetric distributed file storage system, its metadata is scattered across all storage nodes in the system, and generally only the traffic of a certain gateway can be limited; that is, when a single tenant uses multiple gateways, the flow limiting effect cannot be achieved, so in actual use, it is necessary to ensure that the Domain Name System (DNS) returns the same gateway Internet Protocol (IP) to the same tenant.

It can be seen that in order to accurately limit tenant traffic, each tenant can only use one gateway; however, due to the limitations of the network bandwidth and central processing unit (CPU) capacity of a single gateway, the single tenant resource utilization rate is low; especially when the tenant is in a large bandwidth, the traffic control method cannot meet the cloud requirements; therefore, it is necessary to provide a method that can perform traffic control in the case of a single tenant with multiple gateways.

Summary of the invention

The present invention provides a flow control method, device, electronic device and computer storage medium; the problem that a symmetrical distributed file storage system cannot perform flow control in a single-tenant multi-gateway situation can be solved.

The technical solution of the present invention is achieved in this way:

The present invention provides a flow control method, which is applied to a first flow control component of a distributed file storage system, and the method comprises:

After the first traffic control component establishes a connection with each gateway, receiving the real-time traffic of each directory reported by each gateway;

Determine the flow limit value of each directory by each gateway according to the real-time flow of each directory reported by each gateway and the preset flow limit value of each directory;

The flow limit value of each directory of each gateway is sent to each gateway, so that each gateway controls the flow of each directory based on the flow limit value of each directory.

In some embodiments, determining the flow limit value of each directory by each gateway according to the real-time flow of each directory reported by each gateway and the preset flow limit value of each directory includes:

Determine the sum of the real-time traffic of each directory according to the real-time traffic of each directory reported by each gateway;

When it is determined that the total real-time traffic of each directory is greater than the preset traffic limit value of each directory, the traffic limit value of each directory by each gateway is determined according to the real-time traffic ratio of each directory reported by each gateway and the preset traffic limit value of each directory.

In some embodiments, the distributed file storage system further includes a second flow control component, and the method further includes:

When the first flow control component stops providing services, the second flow control component is used to establish connections with the respective gateways; and the services provided by the first flow control component are migrated to the second flow control component.

In some embodiments, the method further comprises:

When multiple flow control components of the distributed file storage system stop service and the time without heartbeat response between each gateway and the multiple flow control components exceeds a set time threshold, the preset flow limit value of each directory is sent to each gateway, so that each gateway controls the flow of each directory based on the preset flow limit value of each directory.

In some embodiments, the method further comprises:

When it is determined that the virtual IP corresponding to the first gateway is transferred to the second gateway, the real-time traffic of each directory reported by the remaining gateways is received again; the remaining gateways represent gateways other than the first gateway.

In some embodiments, receiving the real-time traffic of each directory reported by each gateway includes:

The real-time traffic of each directory reported by each gateway is periodically received according to a set interval.

The present invention provides a flow control device, which is applied to a first flow control component of a distributed file storage system. The device comprises a receiving module, a determining module and a control module, wherein:

A receiving module, configured to receive the real-time traffic of each directory reported by each gateway after the first traffic control component establishes a connection with each gateway;

A determination module, configured to determine the flow limit value of each directory by each gateway according to the real-time flow of each directory reported by each gateway and the preset flow limit value of each directory;

The control module is used to send the flow limit value of each directory of each gateway to each gateway, so that each gateway controls the flow of each directory based on the flow limit value of each directory.

The present invention provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, a flow control method provided by one or more of the above-mentioned technical solutions is implemented.

The present invention provides a computer storage medium, wherein the computer storage medium stores a computer program; after the computer program is executed, the flow control method provided by one or more of the above-mentioned technical solutions can be implemented.

The present invention provides a flow control method, device, electronic device and computer storage medium. The method is applied to a first flow control component of a distributed file storage system. The method comprises: after the first flow control component establishes a connection with each gateway, receiving the real-time flow of each directory reported by each gateway; determining the flow limit value of each directory for each gateway according to the real-time flow of each directory reported by each gateway and the preset flow limit value of each directory; sending the flow limit value of each directory for each gateway to each gateway, so that each gateway controls the flow of each directory based on the flow limit value of each directory; thus, the present invention introduces a flow control component in the distributed file storage system, and the flow control component calculates the flow limit value of each gateway for each directory according to the real-time flow of each directory reported by each gateway and the preset flow limit value; further, according to the flow limit value, the flow control of each directory is realized; since each directory corresponds to a tenant, the flow control of the distributed file storage system in the case of single tenant and multiple gateways can be realized; further, since the introduced flow control component is extremely lightweight, the design of the distributed file storage system software is greatly simplified while realizing flow control.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1a is a schematic diagram of the structure of a symmetrical distributed file storage system in the related art;

FIG1b is a flow chart of a flow control method according to an embodiment of the present invention;

FIG2a is a schematic diagram of the structure of a flow control method in a distributed file storage system according to an embodiment of the present invention;

FIG2 b is a schematic diagram 1 of a format for reporting real-time traffic by a gateway according to an embodiment of the present invention;

FIG2c is a second schematic diagram of a format for reporting real-time traffic by a gateway according to an embodiment of the present invention;

3 is a schematic diagram of the composition structure of a flow control device according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention.

Detailed ways

The technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the present invention.

The present invention will be further described in detail below in conjunction with the accompanying drawings and Examples. It should be understood that the embodiments provided herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the embodiments provided below are partial embodiments for implementing the present invention, rather than providing all embodiments for implementing the present invention. In the absence of conflict, the technical solutions recorded in the present invention can be implemented in any combination.

It should be noted that, in the present invention, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a method or device including a series of elements includes not only the elements explicitly stated, but also other elements not explicitly listed, or also includes elements inherent to the implementation of the method or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other related elements (such as steps in a method or units in a device, for example, a unit may be a part of a processor, a part of a program or software, etc.) in the method or device including the element.

The term "and/or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and/or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

For example, the flow control method provided by the present invention includes a series of steps, but the flow control method provided by the present invention is not limited to the recorded steps. Similarly, the flow control device provided by the present invention includes a series of modules, but the flow control device provided by the present invention is not limited to including the modules explicitly recorded, and can also include modules that need to be set up to obtain relevant information or perform processing based on information.

The present invention can be implemented based on electronic devices, where the electronic devices can be thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, small computer systems, and the like.

Electronic devices may be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. In general, program modules may include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems may be implemented in a distributed cloud computing environment where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules may be located on local or remote computing system storage media including storage devices.

In the public cloud scenario, the file storage requested by the tenant corresponds to a directory in the underlying distributed file storage system, and multiple tenants share the overall traffic of the same storage system. If not controlled, the traffic between tenants will affect each other; it can be seen that traffic control has become an essential function for file storage.

In some embodiments of the present invention, the flow control method can be implemented using a processor in a flow control device, and the processor can be at least one of an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a CPU, a controller, a microcontroller, and a microprocessor.

In the embodiment of the present invention, the flow control method can be applied to a symmetrical distributed file storage system.

FIG. 1b is a flow chart of a flow control method according to an embodiment of the present invention. As shown in FIG. 1b , the process may include:

Step 100: After the first traffic control component establishes connections with each gateway, the real-time traffic of each directory reported by each gateway is received.

Exemplarily, the first flow control component can be a high-availability service based on the Raft algorithm distributed on each gateway; here, the Raft algorithm is mainly used in distributed cluster systems, which can ensure high availability and data consistency; the first flow control component is designed to be very lightweight, and it can be responsible for the management of tenant traffic on each gateway.

In one implementation, each gateway may represent a storage gateway deployed in a tenant's local data center and a public cloud; here, the storage gateway may establish a mapping relationship between the client's file data stored in a storage bucket and the directory of a symmetric distributed file storage system to implement backup and distribution of file data; that is, the storage gateway is equivalent to the entrance to the file data storage, through which the tenant can access the corresponding directory of the distributed file storage system using the NFS protocol or the CIFS protocol.

Exemplarily, the implementation method for the first flow control component to establish a connection with each gateway may be that after the first flow control component is started, a connection is established with each gateway according to the port number of each gateway; or after the first flow control component is started, each gateway establishes a connection with the first flow control component according to the port number of the first flow control component.

In an embodiment of the present invention, after determining that the first flow control component establishes a connection with each gateway, the first flow control component receives the real-time flow of each directory reported by each gateway; illustratively, in a symmetric distributed file storage system where gateway 1 and gateway 2 exist, assuming that the real-time flow of directory A on gateway 1 is 5M, and the real-time flow of directory A on gateway 2 is 10M, the first flow control component will receive the real-time flow of directory A reported by gateway 1, which is 5M, and the real-time flow of directory A reported by gateway 2, which is 10M.

In the embodiment of the present invention, since the first flow control component is decoupled from the symmetric distributed file storage system itself, this not only facilitates tenant flow management, but also reduces the maintenance cost of the symmetric distributed file storage system and improves the stability of the system.

In some embodiments, receiving the real-time traffic of each directory reported by each gateway may include: periodically receiving the real-time traffic of each directory reported by each gateway at a set interval.

In one implementation, the value of the set interval can be set according to actual conditions, for example, it can be 1S, 2S, etc., and the embodiment of the present invention is not limited to this.

Exemplarily, when the time interval is 2S, the first traffic control component receives the real-time traffic of each directory reported by each gateway every 2S.

In some embodiments, the distributed file storage system may further include a second flow control component, and the method may further include: when the first flow control component stops service, using the second flow control component to establish a connection with each gateway; and migrating the service provided by the first flow control component to the second flow control component.

Exemplarily, after the first flow control component establishes connections with each gateway, if the first flow control component stops providing services due to various abnormal situations, the services provided by the first flow control component can be migrated to the second flow control component, that is, the second flow control component immediately takes over the above services provided by the first flow control component; that is, a connection is established between the second flow control component and each gateway; then, the second flow control component receives the real-time traffic of each directory reported by each gateway.

In one implementation, multiple flow control components may be introduced into a distributed file storage system, and among these multiple flow control components, only one flow control component needs to be selected for service each time; and when the flow control component stops providing service, any other flow control component may be selected for providing service; here, each gateway may be unaware of the switching of the flow control components.

In some embodiments, the method may further include: when determining that the virtual IP corresponding to the first gateway is transferred to the second gateway, re-receiving the real-time traffic of each directory reported by the remaining gateways; the remaining gateways represent other gateways except the first gateway.

In one implementation, each of the multiple traffic control components provides only one virtual IP for each gateway. When any gateway fails, for example, crashes, the virtual IP corresponding to the gateway will be transferred to other gateways.

Here, the first gateway and the second gateway represent two different gateways in the distributed file storage system; when it is determined that the first gateway fails, the virtual IP corresponding to the first gateway is transferred to other gateways except the first gateway, for example, the second gateway; then, the traffic control component re-receives the real-time traffic of each directory reported by other gateways except the first gateway.

For example, in a symmetric distributed file storage system where gateway 1, gateway 2 and gateway 3 exist, assuming that the real-time traffic of directory B on gateway 1 is 2M, the real-time traffic of directory B on gateway 2 is 3M, and the real-time traffic of directory B on gateway 3 is 10M, if gateway 1 fails, the virtual IP corresponding to gateway 1 can be transferred to gateway 2 or gateway 3.

For example, when the virtual IP corresponding to gateway 1 is transferred to gateway 2, the traffic control component will receive 5M of real-time traffic of directory B reported by gateway 2 and 10M of real-time traffic of directory B reported by gateway 3; when the virtual IP corresponding to gateway 1 is transferred to gateway 3, the traffic control component will receive 3M of real-time traffic of directory B reported by gateway 2 and 12M of real-time traffic of directory B reported by gateway 3.

It can be seen that in the embodiment of the present invention, when the above-mentioned abnormal situation occurs in the symmetric distributed file storage system, that is, when a flow control component or gateway fails, the control of the directory flow can also be achieved.

Step 101: Determine the flow limit value of each directory for each gateway according to the real-time flow of each directory reported by each gateway and the preset flow limit value of each directory.

Here, each directory has a corresponding preset flow limit value; the setting of the preset flow limit value can be set according to the actual application scenario; the preset flow limit values of different directories can be the same or different; the embodiment of the present invention does not limit this.

In one embodiment, a preset traffic limit value for each directory can be pre-set in the first traffic control component; when the first traffic control component receives the real-time traffic of each directory reported by each gateway, the traffic limit value for each directory of each gateway can be determined based on the real-time traffic of each directory reported by each gateway and the preset traffic limit value of each directory.

In some embodiments, the implementation method of determining the traffic limit value of each directory by each gateway based on the real-time traffic of each directory reported by each gateway and the preset traffic limit value of each directory may include: determining the sum of the real-time traffic of each directory based on the real-time traffic of each directory reported by each gateway; when it is determined that the sum of the real-time traffic of each directory is greater than the preset traffic limit value of each directory, determining the traffic limit value of each directory by each gateway based on the ratio of the real-time traffic of each directory reported by each gateway and the preset traffic limit value of each directory.

In one embodiment, taking directory C as an example, the process of the first traffic control component determining the traffic limit value of each gateway for each directory is explained. First, the real-time traffic sum of directory C is determined based on the real-time traffic of directory C reported by each gateway; then, it is determined whether the real-time traffic sum of directory C is greater than the preset traffic limit value of directory C. If not, no operation is required on the traffic of directory C, and only the preset traffic limit value of each directory is returned to each gateway. If yes, it is necessary to limit the traffic of directory C; at this time, the first traffic control component determines the traffic limit value of each gateway for directory C based on the product of the real-time traffic ratio of directory C reported by each gateway and the preset traffic limit value of directory C. Here, the traffic limit values of other directories can be determined by the above steps, which will not be repeated here. For example, the traffic limit value of each gateway for each directory can be calculated by formula (1):

Here, dira represents one of the directories; node(i) represents the i-th gateway, i ranges from 1 to n, and n represents the number of gateways in the symmetric distributed file storage system; req(dira,node(i)) represents the real-time traffic of directory dira on gateway i; It indicates the total real-time traffic of directory dira on n gateways; limit(dira) indicates the preset traffic limit value of directory dira.

For example, when n is 2, that is, when there are gateways 1 and 2 in the symmetric distributed file storage system, assuming that the real-time traffic req(dira,node(1)) of directory dira on gateway 1 is 6M, and the real-time traffic req(dira,node(2)) of directory dira on gateway 2 is 4M, then the total real-time traffic of directory dira can be determined as is 10M. Assuming that the preset traffic limit value limit(dira) of the directory dira is 8M, it can be determined that the total real-time traffic of the directory dira, 10M, is greater than the preset traffic limit value 8M of the directory dira; at this time, the real-time traffic ratio of the directory dira reported by gateway 1 is determined to be 0.6, and the product of the real-time traffic ratio of the directory dira reported by gateway 1, 0.6, and the preset traffic limit value 8M of the directory dira, 4.8M, is used as the traffic limit value resp(dira,node(1)) of gateway 1 to the directory dira. Similarly, the real-time traffic ratio of the directory dira reported by gateway 2 is determined to be 0.4, and the product of the real-time traffic ratio of the directory dira reported by gateway 2, 0.4, and the preset traffic limit value 8M of the directory dira, 3.2M, is used as the traffic limit value resp(dira,node(2)) of gateway 2 to the directory dira.

It can be seen that in the embodiment of the present invention, when the sum of the real-time traffic of a directory at each gateway exceeds the preset traffic limit value of the directory, the traffic is distributed to each gateway in proportion based on the preset traffic limit value, so that accurate traffic control can be achieved.

Step 102: Send the flow limit value of each directory of each gateway to each gateway, so that each gateway controls the flow of each directory based on the flow limit value of each directory.

In one embodiment, after the first traffic control component determines the traffic limit value of each gateway for each directory, it returns the preset traffic limit value of each directory and the traffic limit value of each gateway for each directory to each gateway; at this time, each gateway can control the traffic of each directory based on the traffic limit value of each directory.

In some embodiments, the method may also include: when multiple traffic control components of the distributed file storage system stop service and the time without heartbeat response between each gateway and the multiple traffic control components exceeds a set time threshold, the preset traffic limit value of each directory is sent to each gateway, so that each gateway controls the traffic of each directory based on the preset traffic limit value of each directory.

For example, in the process of each flow control component in multiple flow control components communicating with each gateway, the sender sends a fixed-format message to the receiver according to certain rules (e.g., periodic sending, idle sending, etc.), and the receiver replies with a fixed-format message after receiving the message. If the message is not received within a set time threshold, it is considered that the current connection is disconnected. Here, the sender can be a flow control component or a gateway; when the sender is a flow control component, the receiver is a gateway.

In one implementation, the value of the set interval can be set according to actual conditions, for example, it can be 3 times, 5 times, etc. of the heartbeat cycle, and the embodiment of the present invention is not limited to this.

Exemplarily, when multiple flow control components of a distributed file storage system stop service and the time without heartbeat response between each gateway and the multiple flow control components exceeds a set time threshold, the preset flow limit value of each directory is sent to each gateway, and each gateway performs flow control on each directory based on the preset flow limit value of each directory, thereby ensuring that tenants on each gateway can normally complete the input/output (I/O) of reading and writing directories.

In an embodiment of the present invention, the functions of the first control component may include: maintaining the real-time traffic of each directory on each gateway and persisting the preset traffic limit value of each directory; maintaining the heartbeat between each gateway and the first control component; calculating the traffic limit value of each gateway for each directory, and sending it to each gateway to implement traffic control.

The present invention provides a flow control method, device, electronic device and computer storage medium. The method is applied to a first flow control component of a distributed file storage system. The method comprises: after the first flow control component establishes a connection with each gateway, receiving the real-time flow of each directory reported by each gateway; determining the flow limit value of each gateway for each directory according to the real-time flow of each directory reported by each gateway and the preset flow limit value of each directory; sending the flow limit value of each gateway for each directory to each gateway, so that each gateway controls the flow of each directory based on the flow limit value of each directory; in this way, the method introduces a flow control component in the distributed file storage system, and the flow control component calculates the flow limit value of each gateway for each directory according to the real-time flow of each directory reported by each gateway and the preset flow limit value; further, according to the flow limit value, the flow control of each directory is realized; since each directory corresponds to a tenant, the flow control of the distributed file storage system in the case of single tenant and multiple gateways can be realized; further, since the introduced flow control component is extremely lightweight, the design of the distributed file storage system software is greatly simplified while realizing flow control.

In order to better reflect the purpose of the present invention, based on the above-mentioned embodiment of the present invention, the flow control method in the distributed file storage system is explained; Figure 2a is a structural schematic diagram of the flow control method in the distributed file storage system of an embodiment of the present invention. As shown in Figure 2a, the structure includes: two tenants (tenant 1 and tenant 2), DNS, N gateways (gateway 1 to gateway N), three storage buckets (storage bucket 1, storage bucket 2 and storage bucket 3) and a first flow control component Qosctl; the first flow control component Qosctl establishes a connection with each of the N gateways.

Tenant 1 and tenant 2 obtain the IPs of N gateways from DNS respectively, and find the corresponding gateways 1 to gateway N according to the IPs of N gateways; gateways 1 to gateway N obtain the real-time traffic of each directory from three storage buckets. Here, the process of the traffic control method may include: after the first traffic control component Qosctl is started, it establishes connections with N gateways according to the agreed port respectively; all gateways periodically report the real-time traffic of each directory to the first traffic control component Qosctl, and the format is shown in Figure 2b and Figure 2c; Figure 2b is a schematic diagram of the format of the gateway reporting real-time traffic in an embodiment of the present invention; it can be seen that gateway 1 reports the real-time traffic of the three directories of directory a, directory b and directory c to the first traffic control component Qosctl, wherein the real-time traffic of directory a, directory b and directory c is 1M, 2M and 3M respectively; Figure 2c is a schematic diagram of the format of the gateway reporting real-time traffic in an embodiment of the present invention; it can be seen that gateway N reports the real-time traffic of the three directories of directory a, directory b and directory c to the first traffic control component Qosctl, wherein the real-time traffic of directory a, directory b and directory c is 0M, 1M and 7M respectively.

Exemplarily, the first traffic control component Qosctl calculates the traffic limit values that each gateway needs to set for the three directories based on the real-time traffic of directory a, directory b, and directory c reported by each gateway and the corresponding traffic limit values. The calculation rules are as follows:

If the sum of the real-time traffic of each of the three directories does not exceed the preset traffic limit value of the corresponding directory, no operation is performed, and only the preset traffic limit value of each directory recorded by the first traffic control component Qosctl is returned to each gateway; if the sum of the real-time traffic of a directory in the three directories exceeds the preset traffic limit value of the corresponding directory, the traffic limit value of each gateway for the directory is determined according to the product of the real-time traffic ratio of the directory reported by each gateway and the preset traffic limit value of the corresponding directory; after determining the traffic limit value, the preset traffic limit value and traffic limit value of the directory recorded by the first traffic control component Qosctl are returned to each gateway. After each gateway receives the traffic limit value for the directory, it immediately updates the traffic value of the corresponding directory.

It can be seen that the embodiment of the present invention provides an implementation method for directory flow control that is highly decoupled from the symmetric distributed file storage system itself through the first flow control component Qosctl; this method implements the control of file storage flow in the symmetric distributed file storage system under the single-tenant multi-gateway situation.

FIG3 is a schematic diagram of the composition structure of a flow control device according to an embodiment of the present invention. The device is applied to a first flow control component of a distributed file storage system. As shown in FIG3 , the device includes: a receiving module 300, a determining module 301 and a control module 302, wherein:

The receiving module 300 is used to receive the real-time traffic of each directory reported by each gateway after the first traffic control component establishes a connection with each gateway;

The determination module 301 is used to determine the flow limit value of each directory of each gateway according to the real-time flow of each directory reported by each gateway and the preset flow limit value of each directory;

The control module 302 is used to send the flow limit value of each directory of each gateway to each gateway, so that each gateway controls the flow of each directory based on the flow limit value of each directory.

In some embodiments, the determination module 301 is used to determine the flow limit value of each directory of each gateway according to the real-time flow of each directory reported by each gateway and the preset flow limit value of each directory, including:

Determine the sum of the real-time traffic of each directory according to the real-time traffic of each directory reported by each gateway;

When it is determined that the total real-time traffic of each directory is greater than the preset traffic limit value of each directory, the traffic limit value of each directory by each gateway is determined according to the real-time traffic ratio of each directory reported by each gateway and the preset traffic limit value of each directory.

In some embodiments, the control module 302 is further configured to:

When multiple flow control components of the distributed file storage system stop service and the time without heartbeat response between each gateway and the multiple flow control components exceeds a set time threshold, the preset flow limit value of each directory is sent to each gateway, so that each gateway controls the flow of each directory based on the preset flow limit value of each directory.

In some embodiments, the receiving module 300 is further used for:

When it is determined that the virtual IP corresponding to the first gateway is transferred to the second gateway, the real-time traffic of each directory reported by the remaining gateways is received again; the remaining gateways represent gateways other than the first gateway.

In some embodiments, the receiving module 300 is further configured to receive the real-time traffic of each directory reported by each gateway, including:

The real-time traffic of each directory reported by each gateway is periodically received according to a set interval.

In practical applications, the above-mentioned receiving module 300, determination module 301 and control module 302 can all be implemented by a processor located in an electronic device, and the processor can be at least one of ASIC, DSP, DSPD, PLD, FPGA, CPU, controller, microcontroller, and microprocessor.

In addition, each functional module in this embodiment can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or software functional modules.

If the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment is essentially or the part that contributes to the relevant technology or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) or a processor to perform all or part of the steps of the method of this embodiment. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, and other media that can store program codes.

Specifically, the computer program instructions corresponding to a flow control method in this embodiment can be stored on a storage medium such as a CD, a hard disk, or a USB flash drive. When the computer program instructions corresponding to a flow control method in the storage medium are read or executed by an electronic device, any flow control method in the aforementioned embodiments is implemented.

Based on the same technical concept as the above embodiments, referring to FIG. 4 , an electronic device 400 provided by an embodiment of the present invention is shown, which may include: a memory 401 and a processor 402; wherein,

Memory 401, used for storing computer programs and data;

The processor 402 is used to execute the computer program stored in the memory to implement any flow control method of the foregoing embodiments.

In practical applications, the memory 401 may be a volatile memory, such as RAM; or a non-volatile memory, such as ROM, flash memory, hard disk drive (HDD) or solid-state drive (SSD); or a combination of the above types of memory, and provide instructions and data to the processor 402.

The processor 402 may be at least one of ASIC, DSP, DSPD, PLD, FPGA, CPU, controller, microcontroller, and microprocessor. It is understandable that for different flow control devices, the electronic device used to implement the processor function may also be other, which is not specifically limited in the embodiment of the present invention.

In some embodiments, the functions or modules included in the device provided by the embodiments of the present invention can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

The methods disclosed in the various method embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments.

The features disclosed in the various product embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new product embodiments.

The features disclosed in the various method or device embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program codes.

The present invention is described with reference to the flowchart and/or block diagram of the method, device (system) and computer program product according to the embodiment of the present invention. It should be understood that each process and/or box in the flowchart and/or block diagram, as well as the combination of the process and/or box in the flowchart and/or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one process or multiple processes in the flowchart and/or one box or multiple boxes in the block diagram.

These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.

The above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention.

Claims (8)

1. A flow control method for use in a first flow control component of a distributed file storage system, the method comprising:

after the first flow control component establishes connection with each gateway, receiving real-time flow of each catalogue reported by each gateway;

determining the sum of the real-time flow of each directory according to the real-time flow of each directory reported by each gateway;

when the real-time flow sum of each directory is determined to be larger than the preset flow limit value of each directory, determining the flow limit value of each gateway to each directory according to the product of the real-time flow proportion of each directory and the preset flow limit value of each directory, which are reported by each gateway; each directory corresponds to one tenant; the preset flow limit value is set according to an actual application scene; the real-time flow ratio of the catalogue represents the ratio of the real-time flow of the catalogue to the sum of the real-time flows of the catalogue reported by each gateway;

and sending the flow limit value of each gateway to each directory to each gateway, so that each gateway controls the flow of each directory based on the flow limit value of each directory.

2. The method of claim 1, wherein the distributed file storage system further comprises a second flow control component, the method further comprising:

establishing a connection with the respective gateway using the second flow control component when the first flow control component is out of service; and migrating services provided by the first flow control component to the second flow control component.

3. The method according to claim 1, wherein the method further comprises:

and when the plurality of flow control components of the distributed file storage system are out of service and the time of no heartbeat response between each gateway and the plurality of flow control components exceeds a set time threshold, sending the preset flow limit value of each directory to each gateway, so that each gateway controls the flow of each directory based on the preset flow limit value of each directory.

4. The method according to claim 1, wherein the method further comprises:

when the virtual Internet Protocol (IP) corresponding to the first gateway is determined to be transferred to the second gateway, the real-time flow of each directory reported by the rest gateways is received again; the remaining gateways represent other gateways than the first gateway.

5. The method of claim 1, wherein receiving real-time traffic for each directory reported by the respective gateway comprises:

and periodically receiving the real-time traffic of each catalogue reported by each gateway according to a set interval.

6. A flow control apparatus for use in a first flow control component of a distributed file storage system, the apparatus comprising:

the receiving module is used for receiving the real-time flow of each catalogue reported by each gateway after the first flow control assembly establishes connection with each gateway;

the determining module is used for determining the sum of the real-time flow of each directory according to the real-time flow of each directory reported by each gateway; when the real-time flow sum of each directory is determined to be larger than the preset flow limit value of each directory, determining the flow limit value of each gateway to each directory according to the product of the real-time flow proportion of each directory and the preset flow limit value of each directory, which are reported by each gateway; each directory corresponds to one tenant; the preset flow limit value is set according to an actual application scene; the real-time flow ratio of the catalogue represents the ratio of the real-time flow of the catalogue to the sum of the real-time flows of the catalogue reported by each gateway;

and the control module is used for sending the flow limit value of each gateway to each directory to each gateway so that each gateway controls the flow of each directory based on the flow limit value of each directory.

7. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method of any one of claims 1 to 5 when the program is executed.

8. A computer storage medium having stored thereon a computer program, which when executed by a processor implements the method of any of claims 1 to 5.