[go: up one dir, main page]

CN107547447B - Distributed file system network communication method, device and network communication system - Google Patents

Distributed file system network communication method, device and network communication system Download PDF

Info

Publication number
CN107547447B
CN107547447B CN201710773464.XA CN201710773464A CN107547447B CN 107547447 B CN107547447 B CN 107547447B CN 201710773464 A CN201710773464 A CN 201710773464A CN 107547447 B CN107547447 B CN 107547447B
Authority
CN
China
Prior art keywords
object storage
information
network
storage device
end network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201710773464.XA
Other languages
Chinese (zh)
Other versions
CN107547447A (en
Inventor
王庆海
李彦博
王曙光
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhengzhou Yunhai Information Technology Co Ltd
Original Assignee
Zhengzhou Yunhai Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhengzhou Yunhai Information Technology Co Ltd filed Critical Zhengzhou Yunhai Information Technology Co Ltd
Priority to CN201710773464.XA priority Critical patent/CN107547447B/en
Publication of CN107547447A publication Critical patent/CN107547447A/en
Application granted granted Critical
Publication of CN107547447B publication Critical patent/CN107547447B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

本发明公开了一种分布式文件系统网络通信方法、装置及网络通信系统,该方法包括:读取网络配置文件,获取后端网络的包括前端网络IP信息和后端网络IP信息的IP信息;利用预设IP选择算法,根据对象存储设备信息,从多个对象存储设备中选择出第一预设数量的第一对象存储设备及第二预设数量的第二对象存储设备;将前端网络IP信息发送至第一对象存储设备,将后端网络IP信息发送至第二对象存储设备,以使第一对象存储设备绑定至前端网络IP,第二对象存储设备绑定至后端网络IP。可见,本申请将一部分对象存储设备绑定至前端网络的关联网卡,将后端网络的部分流量转移到前端网络中,实现数据分流,均衡网络带宽,提高带宽利用率。

Figure 201710773464

The invention discloses a distributed file system network communication method, device and network communication system. The method includes: reading a network configuration file, and obtaining IP information of a back-end network including front-end network IP information and back-end network IP information; Using a preset IP selection algorithm, according to the object storage device information, a first preset number of first object storage devices and a second preset number of second object storage devices are selected from a plurality of object storage devices; The information is sent to the first object storage device, and the back-end network IP information is sent to the second object storage device, so that the first object storage device is bound to the front-end network IP, and the second object storage device is bound to the back-end network IP. It can be seen that the present application binds some object storage devices to the associated network cards of the front-end network, and transfers part of the traffic of the back-end network to the front-end network, so as to realize data offload, balance network bandwidth, and improve bandwidth utilization.

Figure 201710773464

Description

Network communication method and device of distributed file system and network communication system
Technical Field
The present invention relates to the field of communications technologies, and in particular, to a network communication method and apparatus for a distributed file system, and a network communication system.
Background
Currently, each sub-module of the distributed file system interacts with each other through a network, and data writing into the system also depends on the network, and data copying between an OSD (Object-based Storage Device) and an OSD is also performed through the network. The utilization of the network, the rate of the network, etc. directly affect the performance of the file system.
Generally, a network structure in a distributed file system is a structure in which a front-end network (Public network) and a back-end network (Cluster network) are separated. The front-end network has the flow of a client, the flow of a monitoring module and the flow of an MDS (Metadata Server, Metadata Server daemon); the back-end network mainly comprises data copy flow, heartbeat flow and the like among OSD nodes. And the server of the distributed file system mostly adopts a multi-network card architecture, and the front-end network and the back-end network are both bound to the corresponding network cards.
However, in practical applications, when data writing pressure in the distributed file system is high, the bandwidth of the ten-gigabit network card bound by the back-end network reaches a peak value, and the bandwidth of the network card bound by the front-end network does not reach the peak value. At this point, the bottleneck in system performance is on the back-end network. That is, when the network transmission flow is large, the bandwidths of the front-end network and the back-end network in the system are unbalanced, and the bandwidth utilization rate is not high.
Disclosure of Invention
The invention aims to provide a network communication method and device of a distributed file system and a network communication system, and aims to solve the problems that the bandwidth of a front-end network and the bandwidth of a back-end network are unbalanced and the bandwidth utilization rate is low when the network transmission flow of the conventional distributed file system is large.
In order to solve the technical problems, the invention provides the following technical scheme:
a distributed file system network communication method comprises the following steps:
reading a network configuration file, and acquiring IP information of a back-end network, wherein the IP information comprises front-end network IP information and back-end network IP information;
selecting a first object storage device with a first preset number and a second object storage device with a second preset number from a plurality of object storage devices according to the object storage device information by using a preset IP selection algorithm;
and sending the front-end network IP information to the first object storage equipment, and sending the back-end network IP information to the second object storage equipment, so that the first object storage equipment is bound to the front-end network IP according to the front-end network IP information, and the second object storage equipment is bound to the back-end network IP according to the back-end network IP information and then carries out network communication.
Optionally, the reading the network configuration file, and the obtaining the IP information of the backend network including the front-end network IP information and the backend network IP information includes:
and reading the network configuration file, and acquiring the IP information of the back-end network, wherein the IP information comprises the front-end network IP information and the back-end network IP information.
Optionally, the selecting, by using a preset IP selection algorithm, a first preset number of first object storage devices and a second preset number of second object storage devices from the plurality of object storage devices according to the object storage device information includes:
according to the ID information in the object storage device information, performing remainder operation on the ID of each object storage device and a preset base number to obtain a remainder;
and taking the object storage device with the remainder being zero as the first object storage device, and taking the object storage device with the remainder being non-zero as the second object storage device.
Optionally, after the sending the front-end network IP information to the first object storage device and the sending the back-end network IP information to the second object storage device, the method further includes:
reading the modified network configuration file, and acquiring the unique IP information of the back-end network;
and sending the unique IP information to the object storage equipment so that the object storage equipment is bound to the back-end network IP corresponding to the unique IP information.
A distributed file system network communication device, comprising:
the first reading module is used for reading the network configuration file and acquiring the IP information of the back-end network, wherein the IP information comprises the front-end network IP information and the back-end network IP information;
the selection module is used for selecting a first preset number of first object storage devices and a second preset number of second object storage devices from the plurality of object storage devices according to the object storage device information by using a preset IP selection algorithm;
the first sending module is used for sending the front-end network IP information to the first object storage device and sending the back-end network IP information to the second object storage device, so that the first object storage device is bound to a front-end network IP according to the front-end network IP information, and the second object storage device is bound to a back-end network IP according to the back-end network IP information and then carries out network communication.
Optionally, the first reading module includes:
and the reading submodule is used for reading the network configuration file and acquiring the IP information of the back-end network, wherein the IP information comprises the front-end network IP information and the back-end network IP information.
Optionally, the selection module comprises:
the remainder taking submodule is used for carrying out remainder taking operation on the ID of each object storage device and a preset base number according to the ID information in the object storage device information to obtain a remainder;
and the screening submodule is used for taking the object storage device with the remainder being zero as the first object storage device and taking the object storage device with the remainder being non-zero as the second object storage device.
Optionally, the method further comprises:
the second reading module is used for reading the modified network configuration file and acquiring the unique IP information of the back-end network;
and the second sending module is used for sending the unique IP information to the object storage equipment so as to enable the object storage equipment to be bound to a back-end network IP corresponding to the unique IP information.
A network communication system is applied to a distributed file system and comprises a front-end network, a back-end network, a plurality of network cards, an object storage device management module and a plurality of object storage devices;
the object storage device management module is used for reading a network configuration file and acquiring IP information of a back-end network, wherein the IP information comprises front-end network IP information and back-end network IP information; selecting a first object storage device with a first preset number and a second object storage device with a second preset number from the plurality of object storage devices according to the object storage device information by using a preset IP selection algorithm; sending the front-end network IP information to the first object storage device, and sending the back-end network IP information to the second object storage device; and the second object storage device performs network communication after being bound to a back-end network IP according to the back-end network IP information.
Optionally, the number of the network cards is two.
The invention provides a network communication method of a distributed file system, which comprises the following steps: reading a network configuration file, and acquiring IP information of a back-end network, wherein the IP information comprises front-end network IP information and back-end network IP information; selecting a first object storage device with a first preset number and a second object storage device with a second preset number from a plurality of object storage devices according to the object storage device information by using a preset IP selection algorithm; and sending the front-end network IP information to the first object storage equipment, and sending the back-end network IP information to the second object storage equipment, so that the first object storage equipment is bound to the front-end network IP according to the front-end network IP information, and the second object storage equipment is bound to the back-end network IP according to the back-end network IP information and then carries out network communication.
Therefore, compared with the prior art that all the object storage devices are bound to the associated network card of the back-end network, the method can transfer part of the flow of the back-end network into the front-end network to realize data distribution. Therefore, when the network transmission flow is large, the bandwidth of the front-end network and the bandwidth of the back-end network can be balanced, the bandwidth utilization rate is improved, the network transmission rate is optimized, and the performance of the distributed file system is improved. The distributed file system network communication device and the network communication system have the beneficial effects.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a schematic flowchart of a specific implementation of a network communication method of a distributed file system according to an embodiment of the present invention;
FIG. 2 is a diagram illustrating a conventional network communication of a distributed file system according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of network communication of a distributed file system according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a distributed file system network communication device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, fig. 1 is a schematic flowchart illustrating an embodiment of a distributed file system network communication method according to the present invention, where the method includes the following steps:
step 101: and reading the network configuration file, and acquiring the IP information of the back-end network, wherein the IP information comprises the front-end network IP information and the back-end network IP information.
It is to be understood that the network configuration file may refer to a network configuration file of a preconfigured distributed file system, which may include network configuration information of a front-end network and a back-end network, that is, it may include front-end network IP address information and back-end network IP address information, that is, a plurality of IP addresses are configured for the back-end network by using the network configuration file.
By reading the network configuration file, the IP information of the front-end network and the back-end network can be obtained, and the back-end network IP information here may specifically include the front-end network IP information and the back-end network IP information. That is, the back-end network IP information in the network configuration file includes a plurality of IP addresses, and some of the plurality of IP addresses are IP addresses of the front-end network and some are IP addresses of the back-end network. For example, the network configuration file includes front-end network IP information specifically: the public _ network is 192.168.0.0/23, and the backend network IP information specifically includes: 188.188.0.0/16, 192.168.0.0/23.
The distributed file system adopts two network card structures, namely, one network card is bound to the front-end network, and the other network card is bound to the back-end network. At this time, the backend network IP address information obtained by reading the network configuration file should include a front-end network IP and a backend network IP. In some embodiments, the process of reading the network configuration file and acquiring the IP information of the backend network including the front-end network IP information and the backend network IP information may specifically be: and reading the network configuration file, and acquiring the IP information of the back-end network, wherein the IP information comprises front-end network IP information and back-end network IP information.
Certainly, when the distributed file system adopts more than two network cards, the process is similar, and details are not described herein.
And 102, selecting a first preset number of first object storage devices and a second preset number of second object storage devices from the plurality of object storage devices according to the object storage device information by using a preset IP selection algorithm.
It should be noted that the object storage device information may specifically refer to ID information of an object storage device of the object storage device, and the ID information may specifically be a certain sequence of numbers.
The preset IP selection algorithm may be used to determine which object storage devices are bound to the front-end network and which object storage devices are bound to the back-end network according to the object storage device information. The algorithm can be set according to actual requirements and conditions, namely, part of the object storage devices are bound to the associated network card of the front-end network for ensuring the balance of the network bandwidth between the front-end network and the back-end network, and the bandwidth occupied by the bound part of the object storage devices does not influence the normal operation of the corresponding functions of the front-end network.
Therefore, the preset IP selection algorithm is mainly used to determine how many object storage devices to select an associated network card of which one object storage device is bound to the front-end network, for example, 5 object storage devices to select an associated network card of which one object storage device is bound to the front-end network. At this time, the specific values of the first preset number and the second preset number may be determined by a preset IP selection algorithm.
In particular, the first object storage device and the second object storage device may be screened according to parity of object storage device IDs. That is, the object storage devices with odd or even IDs are used as the first object storage device, and the object storage devices with even or odd IDs are used as the second object storage device, at this time, the first preset number and the second preset number are equal, that is, half of the object storage devices are bound to the front-end network, and the other half of the object storage devices are bound to the back-end network.
However, binding half of the object storage devices to the associated network card of the front-end network may occupy a large amount of bandwidth of the front-end network, thereby affecting the original functions of the front-end network. Therefore, preferably, a base number may be set, each object storage device performs a remainder operation on the base number, and the first object storage device and the second object storage device are screened according to the remainders. In some embodiments, the selecting, by using the preset IP selection algorithm and according to the object storage device information, a first preset number of first object storage devices and a second preset number of second object storage devices from the plurality of object storage devices may specifically be: according to the ID information in the object storage equipment information, carrying out remainder operation on the ID of each object storage equipment and a preset base number to obtain a remainder; and taking the object storage device with the remainder being zero as a first object storage device, and taking the object storage device with the remainder being non-zero as a second object storage device.
It is understood that the preset base number can be set according to actual requirements and situations, and for example, it can be specifically 3, 5 or 7. And carrying out remainder operation on the ID information of the object storage equipment and a preset base number, so that one object storage equipment is selected to be bound to the associated network card of the front-end network every a plurality of object storage equipment with the preset base number.
For example, there are currently 10 object storage devices, the IDs of which are 1 to 10, respectively, and the preset cardinality is 3. At this time, the remainder is 1 and is not zero by performing a remainder operation on 1 and 3, so that the device 1 is used as the second object storage device, and the remainder is 0 by performing a remainder operation on 3 and 3, so that the device 3 is used as the first object storage device. And in the same way, performing remainder operation on other object storage devices and the base number 3 in sequence. Finally, of the 10 object storage devices, the devices having IDs of 3, 6, and 9 serve as the first object storage device, and the others serve as the second object storage device.
It can be seen that, according to the preset base number, the first object storage device and the second object storage device are selected by using the preset IP selection algorithm, and the values of the first preset number and the second preset number can be determined reasonably, that is, the object storage devices of the reasonable number are selected to be bound to the associated network card of the front-end network, so that data distribution can be realized without occupying too much bandwidth of the front-end network.
Step 103: and sending the front-end network IP information to the first object storage equipment, and sending the back-end network IP information to the second object storage equipment, so that the first object storage equipment is bound to the front-end network IP according to the front-end network IP information, and the second object storage equipment is bound to the back-end network IP according to the back-end network IP information and then carries out network communication.
After determining whether each object storage device is bound to the front-end network or the back-end network, corresponding IP information can be sent to each object storage device, and thus each object storage device can be bound to a corresponding IP address according to the received IP information.
It should be noted that, the first object storage device is bound to an IP network segment (an associated network card of the front-end network) of the front-end network according to the front-end network IP information; and the second object storage device is bound to the back-end network IP (the associated network card of the back-end network) according to the back-end network IP information.
For example, the associated network card of the current front-end network is network card a, and the associated network card of the back-end network is network card B; in the prior art, all object storage devices are bound to the network card B. In this embodiment, a suitable part of the object storage devices is selected and bound to the network card a, and the remaining object storage devices are bound to the network card B. Therefore, the bandwidth flow of the object storage device can be optimized, namely, part of the object storage device utilizes the part of the bandwidth of the network card A to copy data, and the bandwidth pressure of the network card B is relieved. Meanwhile, when the network transmission flow is large, the bandwidths of the network card A and the network card B can be balanced, and the bandwidth utilization rate is improved.
In order to be compatible with the conventional mode, that is, the distributed file system needs to be switched back to the conventional mode, at this time, the cluster can be restarted by simply modifying the network configuration file, so that the conventional mode can be switched. In some embodiments, after sending the front-end network IP information to the first object storage device and sending the back-end network IP information to the second object storage device, the method further includes: reading the modified network configuration file, and acquiring the unique IP information of the back-end network; and sending the unique IP information to the object storage equipment so that the object storage equipment is bound to the back-end network IP corresponding to the unique IP information.
It can be understood that the backend network IP in the original network configuration file is configured as a single IP, and after the cluster is restarted, all the object storage devices are bound to the modified single IP. For example, the modified network configuration file specifically includes:
public_network=192.168.0.0/23,
cluster_network=188.188.0.0/16。
it can be seen that the network configuration file can be switched from the multi-IP mode to the traditional single-IP mode only by simply modifying the network configuration file, and the compatibility is strong.
To better illustrate the differences between the multi-IP approach and the conventional single-IP approach, the following description will be further described with reference to fig. 2 and 3. Fig. 2 is a schematic diagram of a conventional network communication of a distributed file system according to an embodiment of the present invention, and fig. 3 is a schematic diagram of a network communication of a distributed file system according to an embodiment of the present invention.
As shown in fig. 2, the Client interacts with the distributed file system through a Public network; mon (Monitor, monitoring module), MDS, OSD carry on the interaction through Public network, for the order, report state and diffuse osdmap, etc. of responding to Mon specifically; and data copying, heartbeat message processing and the like are carried out between the OSD and the OSD through a Cluster network.
As shown in fig. 3, some OSDs use Public networks for data copy operations. Mon and MDS still interact through a Public network, a part of OSD interacts through the Public network, and a part of OSD interacts through a Cluster network.
As shown in the conventional network structure of fig. 2, when a large amount of data is to be copied between the OSD and the OSD, the Cluster network becomes abnormally busy, and the network card bound by the Cluster network can reach the peak value of the bandwidth of a ten-gigabit network card. By using the network structure shown in fig. 3, the pressure of the Cluster network can be relieved, and the bandwidth of the Public network can be fully utilized, so that more data can be copied between the OSD and the OSD in unit time, and the system performance is improved.
In this embodiment, the method includes: reading a network configuration file, and acquiring IP information of a back-end network, wherein the IP information comprises front-end network IP information and back-end network IP information; selecting a first object storage device with a first preset number and a second object storage device with a second preset number from a plurality of object storage devices according to the object storage device information by using a preset IP selection algorithm; and sending the front-end network IP information to the first object storage equipment, and sending the back-end network IP information to the second object storage equipment, so that the first object storage equipment is bound to the front-end network IP according to the front-end network IP information, and the second object storage equipment is bound to the back-end network IP according to the back-end network IP information and then carries out network communication.
Therefore, compared with the prior art that all the object storage devices are bound to the associated network card of the back-end network, the method can transfer part of the flow of the back-end network into the front-end network to realize data distribution. Therefore, when the network transmission flow is large, the bandwidth of the front-end network and the bandwidth of the back-end network can be balanced, the bandwidth utilization rate is improved, the network transmission rate is optimized, and the performance of the distributed file system is improved.
In the following, the distributed file system network communication apparatus provided in the embodiment of the present invention is introduced, and the distributed file system network communication apparatus described below and the distributed file system network communication method described above may be referred to correspondingly.
Referring to fig. 4, fig. 4 is a schematic structural diagram of a distributed file system network communication device according to an embodiment of the present invention, where the device may include:
the first reading module 41 is configured to read a network configuration file, and acquire IP information of a backend network, where the IP information includes front-end network IP information and backend network IP information;
a selecting module 42, configured to select, according to the object storage device information, a first preset number of first object storage devices and a second preset number of second object storage devices from the multiple object storage devices by using a preset IP selection algorithm;
the first sending module 43 is configured to send the front-end network IP information to the first object storage device, and send the back-end network IP information to the second object storage device, so that the first object storage device binds to the front-end network IP according to the front-end network IP information, and the second object storage device binds to the back-end network IP according to the back-end network IP information and then performs network communication.
In some embodiments, the first reading module 41 may include:
and the reading submodule is used for reading the network configuration file and acquiring the IP information of the back-end network, wherein the IP information comprises front-end network IP information and back-end network IP information.
In some embodiments, the selection module 42 may include:
the remainder taking submodule is used for carrying out remainder taking operation on the ID of each object storage device and a preset base number according to the ID information in the object storage device information to obtain a remainder;
and the screening submodule is used for taking the object storage device with the remainder being zero as a first object storage device and taking the object storage device with the remainder being non-zero as a second object storage device.
In some embodiments, the method may further include:
the second reading module is used for reading the modified network configuration file and acquiring the unique IP information of the back-end network;
and the second sending module is used for sending the unique IP information to the object storage equipment so as to enable the object storage equipment to be bound to the back-end network IP corresponding to the unique IP information.
In this embodiment, the apparatus selects, by using a preset IP selection algorithm, a first object storage device bound to a front-end network IP, and a second object storage device bound to a back-end network IP, that is, a part of object storage devices are bound to an associated network card of the front-end network, and another part of object storage devices are bound to an associated network card of the back-end network. Therefore, when the network transmission flow is large, the bandwidth of the front-end network and the bandwidth of the back-end network can be balanced, the bandwidth utilization rate is improved, the network transmission rate is optimized, and the performance of the distributed file system is improved.
The network communication system provided in this embodiment is described below, and is specifically applied to a distributed file system, which may include a front-end network, a back-end network, a plurality of network cards, an object storage device management module, and a plurality of object storage devices;
the object storage device management module is used for reading a network configuration file and acquiring IP information of a back-end network, wherein the IP information comprises front-end network IP information and back-end network IP information; selecting a first object storage device with a first preset number and a second object storage device with a second preset number from a plurality of object storage devices according to the object storage device information by using a preset IP selection algorithm; sending the front-end network IP information to a first object storage device, and sending the back-end network IP information to a second object storage device; and the second object storage device performs network communication after being bound to the back-end network IP according to the back-end network IP information.
Preferably, the number of network cards is two.
When the number of the network cards is two, one of the network cards is associated with the front-end network, and the other network card is associated with the back-end network. At this time, the first object storage device is bound to the front-end network IP network segment, namely the first object storage device is bound to the network card associated with the front-end network; the second object storage device is bound to the backend network IP network segment, i.e. the second object storage device is bound to a network card associated with the backend network.
It should be noted that similar parts between this embodiment and the above embodiments may be referred to each other, and are not described herein again.
Therefore, compared with the prior art that all the object storage devices are bound to the associated network card of the back-end network, the communication system can transfer part of the flow of the back-end network into the front-end network to realize data distribution. Therefore, when the network transmission flow is large, the bandwidth of the front-end network and the bandwidth of the back-end network can be balanced, the bandwidth utilization rate is improved, the network transmission rate is optimized, and the performance of the distributed file system is improved.
The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The above detailed description describes the network communication method, device and network communication system of the distributed file system provided by the present invention. The principles and embodiments of the present invention are explained herein using specific examples, which are presented only to assist in understanding the method and its core concepts. It should be noted that, for those skilled in the art, it is possible to make various improvements and modifications to the present invention without departing from the principle of the present invention, and those improvements and modifications also fall within the scope of the claims of the present invention.

Claims (6)

1.一种分布式文件系统网络通信方法,其特征在于,包括:1. a distributed file system network communication method, is characterized in that, comprises: 读取网络配置文件,获取后端网络的包括前端网络IP信息和后端网络IP信息的IP信息;Read the network configuration file, and obtain the IP information of the back-end network including the front-end network IP information and the back-end network IP information; 利用预设IP选择算法,根据对象存储设备信息,从多个对象存储设备中选择出第一预设数量的第一对象存储设备及第二预设数量的第二对象存储设备;Using a preset IP selection algorithm, according to the object storage device information, a first preset number of first object storage devices and a second preset number of second object storage devices are selected from a plurality of object storage devices; 将所述前端网络IP信息发送至所述第一对象存储设备,将所述后端网络IP信息发送至所述第二对象存储设备,以使所述第一对象存储设备根据所述前端网络IP信息绑定至前端网络IP,所述第二对象存储设备根据所述后端网络IP信息绑定至后端网络IP后进行网络通信;Send the front-end network IP information to the first object storage device, and send the back-end network IP information to the second object storage device, so that the first object storage device can use the front-end network IP The information is bound to the front-end network IP, and the second object storage device performs network communication after being bound to the back-end network IP according to the back-end network IP information; 所述利用预设IP选择算法,根据对象存储设备信息,从多个对象存储设备中选择出第一预设数量的第一对象存储设备及第二预设数量的第二对象存储设备包括:The using a preset IP selection algorithm to select a first preset number of first object storage devices and a second preset number of second object storage devices from a plurality of object storage devices according to object storage device information includes: 根据所述对象存储设备信息中的ID信息,将各个所述对象存储设备的ID与预设基数作取余运算,得出余数;According to the ID information in the object storage device information, a remainder operation is performed on the ID of each of the object storage devices and a preset base number to obtain a remainder; 将所述余数为零的所述对象存储设备作为所述第一对象存储设备,所述余数非零的所述对象存储设备作为所述第二对象存储设备;using the object storage device with the remainder zero as the first object storage device, and the object storage device with the remainder non-zero as the second object storage device; 在所述将所述前端网络IP信息发送至所述第一对象存储设备,将所述后端网络IP信息发送至所述第二对象存储设备之后还包括:After the sending the front-end network IP information to the first object storage device and sending the back-end network IP information to the second object storage device, the method further includes: 读取修改后网络配置文件,获取所述后端网络的唯一IP信息;Read the modified network configuration file to obtain the unique IP information of the back-end network; 将所述唯一IP信息发送至所述对象存储设备,以使所述对象存储设备绑定至所述唯一IP信息对应的后端网络IP。The unique IP information is sent to the object storage device, so that the object storage device is bound to the back-end network IP corresponding to the unique IP information. 2.根据权利要求1所述的分布式文件系统网络通信方法,其特征在于,所述读取网络配置文件,获取后端网络的包括前端网络IP信息和后端网络IP信息的IP信息包括:2. distributed file system network communication method according to claim 1, is characterized in that, described reading network configuration file, the IP information that obtains back-end network and comprises front-end network IP information and back-end network IP information comprises: 读取所述网络配置文件,获取所述后端网络的包括一个所述前端网络IP信息和一个后端网络IP信息的所述IP信息。The network configuration file is read to obtain the IP information of the back-end network including one front-end network IP information and one back-end network IP information. 3.一种分布式文件系统网络通信装置,其特征在于,包括:3. A distributed file system network communication device, characterized in that, comprising: 第一读取模块,用于读取网络配置文件,获取后端网络的包括前端网络IP信息和后端网络IP信息的IP信息;The first reading module is used to read the network configuration file, and obtain the IP information of the back-end network including the front-end network IP information and the back-end network IP information; 选择模块,用于利用预设IP选择算法,根据对象存储设备信息,从多个对象存储设备中选择出第一预设数量的第一对象存储设备及第二预设数量的第二对象存储设备;The selection module is configured to use a preset IP selection algorithm to select a first preset number of first object storage devices and a second preset number of second object storage devices from a plurality of object storage devices according to the object storage device information ; 第一发送模块,用于将所述前端网络IP信息发送至所述第一对象存储设备,将所述后端网络IP信息发送至所述第二对象存储设备,以使所述第一对象存储设备根据所述前端网络IP信息绑定至前端网络IP,所述第二对象存储设备根据所述后端网络IP信息绑定至后端网络IP后进行网络通信;a first sending module, configured to send the front-end network IP information to the first object storage device, and send the back-end network IP information to the second object storage device, so that the first object storage device The device is bound to the front-end network IP according to the front-end network IP information, and the second object storage device performs network communication after being bound to the back-end network IP according to the back-end network IP information; 所述选择模块包括:The selection module includes: 取余子模块,用于根据所述对象存储设备信息中的ID信息,将各个所述对象存储设备的ID与预设基数作取余运算,得出余数;The remainder submodule is used to perform remainder operation on the ID of each described object storage device and the preset base number according to the ID information in the object storage device information, and obtain the remainder; 筛选子模块,用于将所述余数为零的所述对象存储设备作为所述第一对象存储设备,所述余数非零的所述对象存储设备作为所述第二对象存储设备;A screening submodule, configured to use the object storage device with the remainder zero as the first object storage device, and the object storage device with the remainder non-zero as the second object storage device; 还包括:Also includes: 第二读取模块,用于读取修改后网络配置文件,获取所述后端网络的唯一IP信息;The second reading module is used to read the modified network configuration file and obtain the unique IP information of the back-end network; 第二发送模块,用于将所述唯一IP信息发送至所述对象存储设备,以使所述对象存储设备绑定至所述唯一IP信息对应的后端网络IP。The second sending module is configured to send the unique IP information to the object storage device, so that the object storage device is bound to the back-end network IP corresponding to the unique IP information. 4.根据权利要求3所述的分布式文件系统网络通信装置,其特征在于,所述第一读取模块包括:4. The distributed file system network communication device according to claim 3, wherein the first reading module comprises: 读取子模块,用于读取所述网络配置文件,获取所述后端网络的包括一个所述前端网络IP信息和一个后端网络IP信息的所述IP信息。The reading submodule is configured to read the network configuration file, and obtain the IP information of the back-end network including one front-end network IP information and one back-end network IP information. 5.一种网络通信系统,应用于分布式文件系统,其特征在于,包括前端网络、后端网络、多个网卡、对象存储设备管理模块及多个对象存储设备;5. A network communication system, applied to a distributed file system, characterized in that it comprises a front-end network, a back-end network, a plurality of network cards, an object storage device management module and a plurality of object storage devices; 所述对象存储设备管理模块用于读取网络配置文件,获取后端网络的包括前端网络IP信息和后端网络IP信息的IP信息;利用预设IP选择算法,根据对象存储设备信息,从多个所述对象存储设备中选择出第一预设数量的第一对象存储设备及第二预设数量的第二对象存储设备;将所述前端网络IP信息发送至所述第一对象存储设备,将所述后端网络IP信息发送至所述第二对象存储设备;以使所述第一对象存储设备根据所述前端网络IP信息绑定至前端网络IP,所述第二对象存储设备根据所述后端网络IP信息绑定至后端网络IP后进行网络通信;其中,所述对象存储设备管理模块具体用于根据所述对象存储设备信息中的ID信息,将各个所述对象存储设备的ID与预设基数作取余运算,得出余数;将所述余数为零的所述对象存储设备作为所述第一对象存储设备,所述余数非零的所述对象存储设备作为所述第二对象存储设备;所述对象存储设备管理模块还用于读取修改后网络配置文件,获取所述后端网络的唯一IP信息;将所述唯一IP信息发送至所述对象存储设备,以使所述对象存储设备绑定至所述唯一IP信息对应的后端网络IP。The object storage device management module is used to read the network configuration file, and obtain the IP information of the back-end network including the front-end network IP information and the back-end network IP information; using a preset IP selection algorithm, according to the object storage device information, from multiple A first preset number of first object storage devices and a second preset number of second object storage devices are selected from the object storage devices; the front-end network IP information is sent to the first object storage device, Send the back-end network IP information to the second object storage device; so that the first object storage device is bound to the front-end network IP according to the front-end network IP information, and the second object storage device is based on the front-end network IP information. After the back-end network IP information is bound to the back-end network IP, network communication is performed; wherein, the object storage device management module is specifically configured to, according to the ID information in the object storage device information, A remainder operation is performed between the ID and the preset base to obtain a remainder; the object storage device with the remainder zero is used as the first object storage device, and the object storage device with the remainder non-zero is used as the first object storage device Two object storage devices; the object storage device management module is further configured to read the modified network configuration file to obtain the unique IP information of the back-end network; send the unique IP information to the object storage device, so that the The object storage device is bound to the back-end network IP corresponding to the unique IP information. 6.根据权利要求5所述网络通信系统,其特征在于,所述网卡的数量为两个。6 . The network communication system according to claim 5 , wherein the number of the network cards is two. 7 .
CN201710773464.XA 2017-08-31 2017-08-31 Distributed file system network communication method, device and network communication system Active CN107547447B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710773464.XA CN107547447B (en) 2017-08-31 2017-08-31 Distributed file system network communication method, device and network communication system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710773464.XA CN107547447B (en) 2017-08-31 2017-08-31 Distributed file system network communication method, device and network communication system

Publications (2)

Publication Number Publication Date
CN107547447A CN107547447A (en) 2018-01-05
CN107547447B true CN107547447B (en) 2021-06-29

Family

ID=60958074

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710773464.XA Active CN107547447B (en) 2017-08-31 2017-08-31 Distributed file system network communication method, device and network communication system

Country Status (1)

Country Link
CN (1) CN107547447B (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106412039A (en) * 2016-09-19 2017-02-15 郑州云海信息技术有限公司 Distributed storage network integration method and system
CN106775459A (en) * 2016-11-28 2017-05-31 深圳市中博睿存科技有限公司 A kind of safe distribution file system framework based on FT 1500A processors
CN107070972A (en) * 2016-12-30 2017-08-18 中国银联股份有限公司 A kind of distributed document processing method and processing device

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7822841B2 (en) * 2007-10-30 2010-10-26 Modern Grids, Inc. Method and system for hosting multiple, customized computing clusters
CN104468388A (en) * 2014-11-04 2015-03-25 浪潮电子信息产业股份有限公司 Linux system based test method for network card load balancing
US10003649B2 (en) * 2015-05-07 2018-06-19 Dell Products Lp Systems and methods to improve read/write performance in object storage applications

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106412039A (en) * 2016-09-19 2017-02-15 郑州云海信息技术有限公司 Distributed storage network integration method and system
CN106775459A (en) * 2016-11-28 2017-05-31 深圳市中博睿存科技有限公司 A kind of safe distribution file system framework based on FT 1500A processors
CN107070972A (en) * 2016-12-30 2017-08-18 中国银联股份有限公司 A kind of distributed document processing method and processing device

Also Published As

Publication number Publication date
CN107547447A (en) 2018-01-05

Similar Documents

Publication Publication Date Title
CN109640116B (en) Cloud mobile phone network video coding method and server
US10545914B2 (en) Distributed object storage
CN111432158B (en) Load balancing method and device for media nodes, storage medium and electronic device
CN106790420B (en) A kind of more session channel method for building up and system
CN107105050B (en) Storage and downloading method and system for service objects
CN115913937B (en) Container multi-network card network configuration method, device, equipment and storage medium
CN111163130B (en) Network service system and data transmission method thereof
CN114157633B (en) Message forwarding method and device
US10067719B1 (en) Methods and systems for storing and accessing data in a distributed data storage system
CN106330610A (en) A load balancing processing system and method
CN105095023A (en) Cloud host establishing device and method and calculating equipment
CN103731472A (en) Method and system for downloading files by adopting P2P technology and client-side
CN111131470B (en) Terminal device, data processing method thereof and data processing system
JP2018521583A (en) Application implementation method and service controller
WO2020119699A1 (en) Resource publishing method and apparatus in internet of things, device, and storage medium
CN107343037B (en) Data transmission method and device for distributed storage system
CN107547447B (en) Distributed file system network communication method, device and network communication system
CN109479214B (en) A load balancing method and related device
CN106790689B (en) Node recommendation method, server and client based on peer-to-peer network
CN110139119B (en) P2P audio live broadcast distribution method, device and storage medium of digital broadcast system
WO2024255247A1 (en) Domain name management method and related device
CN103685469A (en) P2P-based file download method and system, and client terminal
CN115086226B (en) A method and system for establishing anonymous links in anonymous networks
CN115277816A (en) Service adaptation method, device, system and computer readable medium
US20120051261A1 (en) Determination of bypass zones from network configuration settings

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant