[go: up one dir, main page]

CN100476742C - Load Balancing Method Based on Object Storage Device - Google Patents

Load Balancing Method Based on Object Storage Device Download PDF

Info

Publication number
CN100476742C
CN100476742C CNB2007100515099A CN200710051509A CN100476742C CN 100476742 C CN100476742 C CN 100476742C CN B2007100515099 A CNB2007100515099 A CN B2007100515099A CN 200710051509 A CN200710051509 A CN 200710051509A CN 100476742 C CN100476742 C CN 100476742C
Authority
CN
China
Prior art keywords
load
device node
information
copy
node
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CNB2007100515099A
Other languages
Chinese (zh)
Other versions
CN101013387A (en
Inventor
王芳
冯丹
施展
陈亮
彭万利
岳银亮
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huazhong University of Science and Technology
Original Assignee
Huazhong University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huazhong University of Science and Technology filed Critical Huazhong University of Science and Technology
Priority to CNB2007100515099A priority Critical patent/CN100476742C/en
Publication of CN101013387A publication Critical patent/CN101013387A/en
Application granted granted Critical
Publication of CN100476742C publication Critical patent/CN100476742C/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

基于对象存储设备的负载平衡方法,属于计算机存储技术领域,目的在于通过合理调度I/O负载和热点数据迁移,使系统负载均衡分布于各存储节点间,以充分发挥各高性能存储设备节点的性能优势。本发明顺序包括:(1)主动负载检测步骤;(2)设备负载统计步骤;(3)对象迁移和副本管理步骤;(4)对象属性扩展步骤;(5)I/O请求处理步骤。本发明对对象存储设备(OSD)的SCSI协议标准进行了扩展,借助对象存储模式的优点,为I/O调度提供决策依据,充分利用所有存储设备节点的处理能力,达到负载均衡,降低存储系统响应时间,提高存储系统吞吐率。

Figure 200710051509

The load balancing method based on the object storage device belongs to the field of computer storage technology, and the purpose is to make the system load evenly distributed among the storage nodes through reasonable scheduling of I/O load and hot data migration, so as to give full play to the performance of each high-performance storage device node. performance advantage. The sequence of the present invention includes: (1) active load detection step; (2) device load statistics step; (3) object migration and copy management step; (4) object attribute expansion step; (5) I/O request processing step. The present invention expands the SCSI protocol standard of the object storage device (OSD), provides decision-making basis for I/O scheduling by virtue of the advantages of the object storage mode, makes full use of the processing capabilities of all storage device nodes, achieves load balance, and reduces storage system Response time, improve storage system throughput.

Figure 200710051509

Description

Balancing method of loads based on object storage equipment
Technical field
The invention belongs to computer memory technical field, be specifically related to a kind of balancing method of loads based on object storage equipment.
Background technology
Along with the develop rapidly of the computer technology and the network communications technology, global information storage to be to surpass 30% speed sharp increase every year, and the Data Growth rate on the hard disk of being stored in is especially up to 114%.In the face of the explosive increase of data volume and people demand to the aspects such as capacity, security, extensibility and availability of storage system, traditional direct memory module (Direct Access Storage, DAS) seemed unable to do what one wishes, this makes the network storage become the inevitable trend of technical field of memory.The main framework of current network storage has attached net storage (NAS, Network Attached Storage) and storage area network (SAN, Storage Area Network).Though NAS and SUN have solved many problems of direct storage, the limitation of himself is also arranged, can not satisfy the requirement of memory technology development to a certain extent fully.Object-oriented memory technology (OBS, Object Basedstorage) is data-centered network storage pattern, adopted object as BTU, data storage is separated with metadata management, broken through the file-sharing restriction of data routing bottleneck common among the NAS and SAN, and even better in characteristics such as security, professional platform independence, availability and extensibility, may become the standard that the Next Generation Internet network is stored.
Object storage system (OBSS, Object-Based Storage System) combines object interface and intelligent storage equipment, can reach the storage size of PB level.And along with the expansion of storage system scale and the enhancing of frequency of utilization, the unbalanced phenomenon of load inevitably can manifest between memory node, becomes the bottleneck of total system, when serious even can cause the performance of total system sharply to descend.
Summary of the invention
The invention provides a kind of balancing method of loads based on object storage equipment, purpose is by rational management I/O load and hot spot data migration, system load balancing is distributed between each memory node, to give full play to the performance advantage of each high-performance storage devices node.
A kind of balancing method of loads based on object storage equipment of the present invention, order comprises the steps:
(1) active load detecting step, each device node initiatively carries out the load detecting of self, and load factor is sent to meta data server;
(2) apparatus of load statistic procedure, the load of calculating each device node according to method for normalizing, and according to loading condition each device node is sorted;
(3) object migration and replica management step, the device node that load value is surpassed mobility threshold moves and replica management as the focus object and to it;
(4) object properties spread step, property pages to OSD SCSI consensus standard is expanded, a self-defined load attribute page or leaf, these five attribute items of predefine load value, temperature, primary copy judgement, copy information and load weights are preserved load information, the migration information of each device node of needs in the mode of object properties;
(5) I/O request processing step, according to the distributed intelligence of object, scheduling I/O request between each device node makes each device node load balancing.
Described balancing method of loads based on object storage equipment is characterized in that described active load detecting sequence of steps comprises following process:
(1) load information of each device node reading system queue length and disk, CPU, internal memory and network from/proc Virtual File System;
(2) calculate each device node load factor;
(3) each device node load factor is judged to delay time 10 seconds in load factor≤30; If delayed time 30 seconds in 30<load factor≤60; Delayed time 60 seconds in load factor>60;
(4) load information of reading system queue length and disk, CPU, internal memory and network from/proc Virtual File System once more;
(5) send each device node load factor to meta data server, turn over journey (1).
Described balancing method of loads based on object storage equipment is characterized in that described apparatus of load statistic procedure comprises following process in proper order:
(1) definition load factor is defined as each device node load factor LOAD:
LOAD=W 1*Lrql+W 2*Ldisk+W 3*Lcpu+W 4*Lmem+W 5*Lnet
Carry out the normalization management, Lrql, Ldisk, Lcpu, Lmem and Lnet are respectively system queue length, disk load, cpu load, internal memory load and the offered load of object storage equipment, W 1, W 2, W 3, W 4, W 5Be the weights of correspondence, and ∑ W i(i=1,2,3,4,5)=1;
(2) setting the initial load weights, serves as according to W is set with the load value that obtains behind the load full test of file system standard test procedure Iozone to each device node iInitial value;
(3) judge the individual event load state;
(4) according to load information correction load weights W i, continuous five detection individual event load values are all greater than 80 or less than 20, and then its corresponding weights increase or reduce by 10%, otherwise constant;
(5) calculate each device node load factor;
(6) send each device node load factor to meta data server;
(7) meta data server sorts by each device node load factor size, safeguards the ascending order device queue, turns over journey (3).
Described balancing method of loads based on object storage equipment is characterized in that described object migration and replica management sequence of steps comprise following process:
(1) set mobility threshold, this mobility threshold is the critical condition of moving;
(2) detect load, detect each device node load, and add up the load average of each device node nearest ten times;
(3) judge the load average, nearest ten load averages of each device node are judged, if this average greater than mobility threshold, turns over journey (4), otherwise turns over journey (5);
(4) meta data server, moves object as the migration target from the contiguous device node of selecting the load factor minimum; Meta data server upgraded copy information after migration was finished;
(5) if nearest ten load averages less than half of threshold value, the deletion copy, the upgating object metadata information, otherwise, turn over journey (2).
Described balancing method of loads based on object storage equipment is characterized in that described object properties spread step comprises following process in proper order:
(1) determines extended mode, determine to use interim attribute extended mode;
(2) stipulate the attribute page number of each device node, the attribute page number of each device node for C000 0000h to EFFF FFFFh, i.e. D=C000 0000h, then D+5h represents C000 0005h;
(3) definition load attribute page structure, definition load value, temperature, primary copy judgement, copy information and these five attribute items of load weights.
Described balancing method of loads based on object storage equipment is characterized in that described I/O request processing step comprises following process in proper order:
(1) judge user's request type, read request is carried out process (2); Write request is carried out process (3); Update request is carried out process (4);
(2) judge whether the object of asking exists copy, exists then to select the underload copy to read, and turns over journey (5); Otherwise directly read, turn over journey (5) from this object;
(3) according to file size decision burst number N, the N that load factor is minimum in a meta data server selective system device node carries out write operation, turns over journey (5);
(4) after the primary copy information of judgement object, the object master is originally upgraded operation, after renewal was finished, all copies of upgating object carried out next process again;
(5) update metadata information and object properties information.
The present invention is applicable to object-based large-scale storage systems, has following characteristics:
(1) object storage equipment initiatively obtains every load, sends load value;
(2) weights of every load are according to measured data, and can dynamically revise;
(3) judge that by load value attribute, temperature attribute, the primary copy of EXPANDING DISPLAY AREA attribute and copy information attribute provide decision information;
(4) expanding to flow process that meta data server carries out I/O operation by load detecting and attribute is optimized scheduling decision-making foundation is provided;
(5) realize the load migration of hot spot data according to dynamic load information;
The present invention not only can realize the load balancing of system, is equally applicable to the application that the backup selection of significant data and the backup node selection of meta data server etc. need Dynamic Selection equipment.
Description of drawings
Fig. 1 is a FB(flow block) of the present invention;
Fig. 2 is an active load detecting flow chart of steps of the present invention;
Fig. 3 is a load statistics flow chart of steps of the present invention;
Fig. 4 is object migration of the present invention and replica management flow chart of steps;
Fig. 5 is an object properties spread step process flow diagram of the present invention;
Fig. 6 is an extended attribute page structure synoptic diagram of the present invention;
Fig. 7 is an I/O request processing step process flow diagram of the present invention.
Embodiment
The present invention is described in more detail below in conjunction with drawings and Examples.
Fig. 1 is a FB(flow block) of the present invention.The present invention includes: (1) is the load detecting step initiatively; (2) apparatus of load statistic procedure; (3) object migration and replica management step; (4) object properties spread step; (5) I/O request processing step.
Fig. 2 is an active load detecting flow chart of steps of the present invention.The load information that from/proc Virtual File System, reads; The computational load factor is judged each device node load factor, if load factor is higher, then devices illustrated is in busy condition, should reduce the number of times of load statistics, otherwise devices illustrated is in idle condition, can increase the load statistics number of times, set load factor≤30, delay time 10 seconds; If delayed time 30 seconds in 30<load factor≤60; Delayed time 60 seconds in load factor>60; The load information of reading system queue length, disk, CPU, internal memory and network from/proc Virtual File System once more; Send the load factor of each device node to meta data server; Judge time-delay according to load factor once more, so circulation.
Fig. 3 is a load statistics flow chart of steps of the present invention.The load factor LOAD of memory device is defined as:
LOAD=W 1*Lrql+W 2*Ldisk+W 3*Lcpu+W 4*Lmem+W 5*Lnet
Carry out the normalization management, Lrql, Ldisk, Lcpu, Lmem and Lnet are respectively system queue length, disk load, cpu load, internal memory load and the offered load of object storage equipment, W 1, W 2, W 3, W 4, W 5Be the weights of correspondence, and ∑ W i(i=1,2,3,4,5)=1; During the computational load value five loads all are converted to 100 to be the shared ratio of unit, by ∑ W i=1 as can be known load value LOAD more than or equal to zero smaller or equal to 100.With the average that obtains system request queue length behind the load full test of Iozone to each device node is 3.280, maximal value is 7.806, the utilization rate average of CPU, internal memory, the network bandwidth is respectively 33.0781%, 58.0078%, 24.4531%, magnetic disc i/o handling capacity average is 46.1016MB/S, and the utilization rate average is 76.836%.Selected W1 is 0.4, can get one group of load initial weight [0.4,0.2406,0.1032,0.1812,0.075] with universality; After judging the individual event load state, according to load information correction load weights W i, if a certainly load on that load value is all greater than 80 or less than 20 in continuous detect for five times, then its corresponding weights increase or reduce by 10%, otherwise constant; Send each device node load factor to meta data server after calculating each device node load factor; Meta data server sorts by each device node load factor size, safeguards the ascending order device queue.
Fig. 4 is object migration of the present invention and replica management flow chart of steps.The focus object definition is for frequently carrying out the object of read/write operation, and the frequent read/write of object must cause the lifting of corresponding stored device node load, and each device node is set mobility threshold, and this mobility threshold is the critical condition of moving; Detect the loading condition of each device node, and statistics, calculate the load average of each device node nearest ten times; Judge the load average, each device node judged that if this average greater than mobility threshold, is carried out the migration of object, meta data server, moves object as the migration target from the contiguous device node of selecting the load factor minimum during object migration; Meta data server upgraded copy information after migration was finished; If nearest ten load averages are less than half of threshold value, deletion copy, upgating object metadata information.
The focus object definition is for frequently carrying out the object of read/write operation, and threshold value is the critical condition of moving, and temperature is the frequent degree of Object Operations, and to the read/write operation each time of object, temperature adds does not have read/write operation in 1,1 minute, and temperature subtracts 1; Detect temperature, if temperature is greater than mobility threshold, then meta data server from the nearby device node, selects the load factor minimum node as moving target, it is moved; Meta data server upgraded copy information after migration was finished; If read operation is moved to the object copies of underloading node with read request, if write operation then needs to upgrade all copies; Detect temperature,, then delete copy, update metadata information as if temperature half less than threshold value.
Fig. 5 is an object properties spread step process flow diagram of the present invention.Function is preserved in the mode of object properties for load information, migration information that the present invention is needed.In OSD SCSI consensus standard, object properties are described with many property pages, each property pages is made up of many concrete attribute items again, property pages is determined by attribute page number Page Number, the attribute item is to there being attribute AttributeNumber, like this, a concrete attribute item comes index with two tuples (Page Number, Attribute Number).Object properties can be divided into permanent attribute and interim attribute according to the relative length of life period, and what each the attribute item among the present invention in the load attribute page or leaf all embodied is interior object accesses feature of a period of time, is fit to describe with interim attribute; The page number of specified loads property pages for C000 0000h to C000 FFFFh, i.e. D=C000 0000h, then D+5h represents C000 0005h.
Fig. 6 is a load attribute page structure of the present invention.Property pages to OSD SCSI consensus standard is expanded, a self-defined load attribute page or leaf, according to demand of the present invention, these five attribute items of predefine load value, temperature, primary copy judgement, copy information and load weights, all the other attribute items keep does expansion from now on.
Load value attribute item is preserved present device load statistics step gained load factor, is described by integer data, accounts for four bytes.
Temperature attribute item is preserved temperature, is described by integer data, accounts for four bytes, and the big expression operation of temperature is frequent, because hot spot data is mainly by reading concurrent generation in a large number, so can disperse temperature by replication policy.Temperature is moved to a copy of object on the underloading node after surpassing setting threshold, if temperature is still bigger after once moving, then carries out the migration second time.
Primary copy judges that the attribute item is used for judging to liking originally main or copy, owing to only carry out single judgement, so be set to character type data, take a son and saves.Write operation only carries out on leading originally, follows latest copy information after finishing again.
Copy information attribute item has been preserved object and whether has been had copy, has information such as how many copies, takies 20 bytes.For because the copy that hot spot data produces if the temperature of hot spot data drops to a certain threshold value after a period of time, according to principle of locality, can think that focus disappears, can delete copy.
Load weights attribute item has been preserved the load weights of five load correspondences of the present invention's statistics, and each weights accounts for four bytes by a floating type variable description, and five loads take 20 bytes, and keeps 20 bytes do expansion uses.
Fig. 7 is an I/O request processing step process flow diagram of the present invention.In the object storage system, the generation of hot spot data is almost concurrent and produce by a large amount of read operations, the read/write load can separate after keeping a plurality of copies in the system, read operation can be carried out on any copy, can only be applied to originally main and upgrade operation, and all read operations need to lose efficacy when upgrading operation and carrying out, and treat that main this renewal operation is finished and more after the latest copy, read operation just can restart.Can there be a plurality of copies in significant data because of the requirement of reliability, copy exist for load balance and utilize object properties information that good application foundation is provided in a large number.After the I/O request arrives, at first judge request type, request type can be divided into read request, write request and update request.When I/O is read request, judge whether to exist copy, exist under the copy situation, meta data server is found out the minimum equipment of load in the object storage equipment that has copy according to the present load record, and the I/O request is guided to this equipment; Otherwise, do not consider load state, directly from the memory device that has object, read.When I/O was write request, meta data server at first according to the burst number N of file size decision objects, was selected the minimum N of load the memory node as object, storage object according to present load record and historical load record again from all devices node.When I/O asked to update request, the renewal operation can only be applied to originally main, carried out the renewal of copy after main renewal operation is originally finished again.Update metadata and object belonged to part information after all operations was finished.

Claims (1)

1.一种基于对象存储设备的负载平衡方法,顺序包括下述步骤:1. A load balancing method based on an object storage device, the sequence comprising the steps of: (1)主动负载检测步骤,各设备节点主动进行自身的负载检测,并将负载因子发送到元数据服务器;顺序包括下述过程:(1) Active load detection step, each device node actively performs its own load detection, and sends the load factor to the metadata server; the sequence includes the following processes: (1.1)各设备节点从/proc虚拟文件系统中读取系统队列长度以及磁盘、CPU、内存和网络的负载信息;(1.1) Each device node reads the system queue length and load information of disk, CPU, memory and network from the /proc virtual file system; (1.2)计算各设备节点负载因子;(1.2) Calculate the load factor of each equipment node; (1.3)对各设备节点负载因子进行判断,负载因子≤30,延时10秒;若30<负载因子≤60,延时30秒;负载因子>60,延时60秒;(1.3) Judging the load factor of each device node, if the load factor is ≤30, the delay is 10 seconds; if 30<load factor≤60, the delay is 30 seconds; if the load factor is >60, the delay is 60 seconds; (1.4)再次从/proc虚拟文件系统中读取系统队列长度以及磁盘、CPU、内存和网络的负载信息;(1.4) read the system queue length and the load information of disk, CPU, memory and network from the /proc virtual file system again; (1.5)向元数据服务器发送各设备节点负载因子,转过程(1.1);(1.5) Send the load factor of each device node to the metadata server, and turn to the process (1.1); (2)设备负载统计步骤,按照归一化方法计算各设备节点的负载,并依据负载情况对各设备节点进行排序;顺序包括下述过程:(2) The equipment load statistics step is to calculate the load of each equipment node according to the normalization method, and sort each equipment node according to the load situation; the order includes the following processes: (2.1)定义负载因子,将各设备节点负载因子LOAD定义为:LOAD=W1*Lrql+W2*Ldisk+W3*Lcpu+W4*Lmem+W5*Lnet进行归一化管理,Lrql、Ldisk、Lcpu、Lmem和Lnet分别为对象存储设备的系统队列长度、磁盘负载、CPU负载、内存负载和网络负载,W1、W2、W3、W4、W5为对应的权值,且∑Wi(i=1,2,3,4,5)=1;(2.1) Define the load factor, define the load factor LOAD of each device node as: LOAD=W 1 *Lrql+W 2 *Ldisk+W 3 *Lcpu+W 4 *Lmem+W 5 *Lnet for normalized management, Lrql , Ldisk, Lcpu, Lmem, and Lnet are the system queue length, disk load, CPU load, memory load, and network load of the object storage device respectively, and W 1 , W 2 , W 3 , W 4 , and W 5 are the corresponding weights, And ΣW i (i=1, 2, 3, 4, 5)=1; (2.2)设定初始负载权值,以文件系统标准测试程序Iozone对各设备节点的负载全面测试后得到的负载值为依据设置Wi初始值;(2.2) Initial load weight is set, and the load value obtained after the load comprehensive test of each device node with the file system standard test program Iozone is based on setting the W i initial value; (2.3)判断单项负载状况;(2.3) Judging the individual load status; (2.4)根据负载信息修正负载权值Wi,连续五次检测单项负载值均大于80或小于20,则其对应权值增加或降低10%,否则不变;(2.4) Correct the load weight value W i according to the load information, and if the single load value is greater than 80 or less than 20 for five consecutive detections, the corresponding weight value will increase or decrease by 10%, otherwise it will remain unchanged; (2.5)计算各设备节点负载因子;(2.5) Calculate the load factor of each equipment node; (2.6)向元数据服务器发送各设备节点负载因子;(2.6) Send each device node load factor to the metadata server; (2.7)元数据服务器按各设备节点负载因子大小进行排序,维护升序设备队列,转过程(2.3);(2.7) The metadata server sorts according to the load factor of each device node, maintains the ascending device queue, and transfers to the process (2.3); (3)对象迁移和副本管理步骤,对负载值超过迁移阈值的设备节点作为热点对象并对其进行迁移和副本管理;顺序包括下述过程:(3) Object migration and copy management steps, the device node whose load value exceeds the migration threshold is used as a hotspot object and is migrated and copy managed; the sequence includes the following processes: (3.1)设定迁移阈值,该迁移阈值为进行迁移的临界条件;(3.1) Setting the migration threshold, which is a critical condition for migration; (3.2)检测负载,检测各设备节点负载,并统计各设备节点最近十次的负载均值;(3.2) Detect the load, detect the load of each device node, and count the last ten load averages of each device node; (3.3)判断负载均值,对各设备节点最近十次负载均值进行判断,若该均值大于迁移阈值,转过程(3.4),否则转过程(3.5);(3.3) Judging the load average value, judging the last ten load average values of each device node, if the average value is greater than the migration threshold, turn to process (3.4), otherwise turn to process (3.5); (3.4)元数据服务器从邻近选择负载因子最小的设备节点作为迁移目标,对对象进行迁移;迁移完成后元数据服务器更新副本信息;(3.4) The metadata server selects the equipment node with the smallest load factor from the neighbors as the migration target, and migrates the object; after the migration is completed, the metadata server updates the copy information; (3.5)若最近十次负载均值小于阈值的一半,删除副本,更新对象元数据信息,否则,转过程(3.2);(3.5) If the last ten load averages are less than half of the threshold, delete the copy and update the object metadata information, otherwise, go to the process (3.2); (4)对象属性扩展步骤,对OSD SCSI协议标准的属性页进行了扩展,自定义一个负载属性页,预定义负载值、热度、主副本判断、副本信息和负载权值这五个属性项,将需要的各设备节点的负载信息、迁移信息以对象属性的方式保存;顺序包括下述过程:(4) The step of object property expansion, which extends the property page of the OSD SCSI protocol standard, customizes a load property page, and pre-defines the five property items of load value, heat, primary copy judgment, copy information and load weight, Save the required load information and migration information of each device node in the form of object attributes; the sequence includes the following process: (4.1)确定扩展方式,确定使用临时属性扩展方式;(4.1) Determine the expansion method, and determine the use of temporary attribute expansion methods; (4.2)规定各设备节点的属性页码,各设备节点的属性页码为C000 0000h到EFFF FFFFh,即D=C000 0000h,则D+5h表示C000 0005h;(4.2) Specify the property page number of each device node, the property page number of each device node is C000 0000h to EFFF FFFFh, that is, D=C000 0000h, then D+5h means C000 0005h; (4.3)定义负载属性页结构,定义负载值、热度、主副本判断、副本信息和负载权值这五个属性项;(4.3) Define the structure of the load property page, and define the five property items of load value, heat, master copy judgment, copy information and load weight; (5)I/O请求处理步骤,依据对象的分布信息,在各设备节点间调度I/O请求,使得各设备节点负载均衡;顺序包括下述过程:(5) The I/O request processing step, according to the distribution information of the object, schedules the I/O request among each device node, so that each device node load is balanced; the sequence includes the following process: (5.1)判断用户请求类型,读请求,进行过程(5.2);写请求,进行过程(5.3);更新请求,进行过程(5.4);(5.1) Judging the type of user request, read request, process (5.2); write request, process (5.3); update request, process (5.4); (5.2)判断请求的对象是否存在副本,存在则选择轻负载副本读取,转过程(5.5);否则直接从该对象读取,转过程(5.5);(5.2) Determine whether there is a copy of the requested object, if it exists, choose a light-load copy to read, and go to the process (5.5); otherwise read directly from the object, go to the process (5.5); (5.3)根据文件大小决定分片数目N,元数据服务器选择系统中负载因子最低的N个设备节点进行写操作,转过程(5.5);(5.3) Determine the number N of fragments according to the file size, and the metadata server selects the N device nodes with the lowest load factor in the system to perform the write operation, and transfer to the process (5.5); (5.4)判断对象的主副本信息后,对对象主本进行更新操作,更新完成后,再更新对象所有副本,进行下一过程;(5.4) After judging the master copy information of the object, update the master copy of the object. After the update is completed, update all copies of the object and proceed to the next process; (5.5)更新元数据信息和对象属性信息。(5.5) Update metadata information and object attribute information.
CNB2007100515099A 2007-02-09 2007-02-09 Load Balancing Method Based on Object Storage Device Expired - Fee Related CN100476742C (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CNB2007100515099A CN100476742C (en) 2007-02-09 2007-02-09 Load Balancing Method Based on Object Storage Device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CNB2007100515099A CN100476742C (en) 2007-02-09 2007-02-09 Load Balancing Method Based on Object Storage Device

Publications (2)

Publication Number Publication Date
CN101013387A CN101013387A (en) 2007-08-08
CN100476742C true CN100476742C (en) 2009-04-08

Family

ID=38700918

Family Applications (1)

Application Number Title Priority Date Filing Date
CNB2007100515099A Expired - Fee Related CN100476742C (en) 2007-02-09 2007-02-09 Load Balancing Method Based on Object Storage Device

Country Status (1)

Country Link
CN (1) CN100476742C (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102541460A (en) * 2010-12-20 2012-07-04 中国移动通信集团公司 Multiple disc management method and equipment

Families Citing this family (43)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101562626B (en) * 2008-04-15 2013-08-07 华为技术有限公司 Method, system and device for medium distribution
CN101442435B (en) * 2008-12-25 2012-04-04 华为技术有限公司 Service data management method and device of distributed system and distributed system
CN101887440B (en) * 2009-05-13 2012-05-30 财团法人资讯工业策进会 Hot spot analysis system and method
CN101610287B (en) * 2009-06-16 2012-03-14 浙江大学 Method for balancing load applied in distributed mass memory system
CN101662495B (en) * 2009-09-16 2012-09-05 成都市华为赛门铁克科技有限公司 Backup method, master server, backup servers and backup system
CN102055650B (en) * 2009-10-29 2012-10-17 华为技术有限公司 Load balancing method and system and management server
CN101854398B (en) * 2010-06-07 2014-12-10 中兴通讯股份有限公司 File copy equilibrium distribution method, device and system
CN102622185B (en) * 2011-01-27 2015-09-23 北京东方广视科技股份有限公司 The method of storage file and storage allocation method in multiple storage unit
CN102117334A (en) * 2011-03-14 2011-07-06 浪潮(北京)电子信息产业有限公司 Method and system for managing file storage in cloud storage system
US8365023B2 (en) * 2011-04-29 2013-01-29 International Business Machines Corporation Runtime dynamic performance skew elimination
BR112014010370B1 (en) * 2011-10-31 2021-08-10 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. APPARATUS AND METHOD FOR TRANSFERRING EVENT DETECTOR PROCESSES
CN103229487B (en) * 2012-12-27 2015-09-09 华为技术有限公司 Partition balancing method, device and server in distributed memory system
CN103049559A (en) * 2012-12-29 2013-04-17 深圳先进技术研究院 Automatic mass data placement method and device
CN103150215B (en) * 2013-02-04 2016-03-02 浙江大学 Fine-grained cpu resource usage forecastings method under virtual environment
CN103246716B (en) * 2013-04-26 2016-02-24 中国传媒大学 Based on object copies efficient management and the system of object cluster file system
CN104144127A (en) * 2013-05-08 2014-11-12 华为软件技术有限公司 Load Balancing Methods and Load Balancers
CN104579996A (en) * 2013-10-17 2015-04-29 中国电信股份有限公司 Cluster load balancing method and system
US9813335B2 (en) * 2014-08-05 2017-11-07 Amdocs Software Systems Limited System, method, and computer program for augmenting a physical system utilizing a network function virtualization orchestrator (NFV-O)
CN103761059B (en) * 2014-01-24 2017-02-08 中国科学院信息工程研究所 Multi-disk storage method and system for mass data management
CN103823639A (en) * 2014-02-19 2014-05-28 曙光信息产业(北京)有限公司 Method and system for migrating objects
CN103916467B (en) * 2014-03-25 2017-02-08 中国科学院计算技术研究所 Load transfer method and system in metadata cluster
CN104202434A (en) * 2014-09-28 2014-12-10 北京奇虎科技有限公司 Node access method and device
JP6403140B2 (en) 2014-10-22 2018-10-10 華為技術有限公司Huawei Technologies Co.,Ltd. Method, controller, and system for service flow control in an object-based storage system
CN104331253B (en) * 2014-10-30 2017-12-15 浪潮电子信息产业股份有限公司 Calculation method for object migration in object storage system
CN105700951B (en) * 2014-11-25 2021-01-26 中兴通讯股份有限公司 Method and device for realizing CPU service migration
CN105740077B (en) * 2016-01-29 2020-02-07 中国联合网络通信集团有限公司 Task allocation method suitable for cloud computing
CN108063780B (en) * 2016-11-08 2021-02-19 中国电信股份有限公司 Method and system for dynamically replicating data
CN107239544A (en) * 2017-06-05 2017-10-10 山东浪潮云服务信息科技有限公司 The implementation method and device of a kind of distributed storage
CN107465729B (en) * 2017-07-25 2020-09-22 北京计算机技术及应用研究所 Virtual node copy adjusting system and method based on access time ranking
CN107688625A (en) * 2017-08-18 2018-02-13 郑州云海信息技术有限公司 A kind of method and system of the storage small documents based on distributed memory system
CN107506491B (en) * 2017-09-26 2021-03-09 苏州浪潮智能科技有限公司 OSD data distribution method and device of distributed file system
CN109936473B (en) * 2017-12-19 2022-04-08 北京华耀科技有限公司 Deep learning prediction-based distributed computing system and operation method thereof
CN108197229A (en) * 2017-12-29 2018-06-22 北京搜狐新媒体信息技术有限公司 The balance method and system of a kind of data in magnetic disk
WO2020000136A1 (en) * 2018-06-25 2020-01-02 Alibaba Group Holding Limited System and method for managing resources of a storage device and quantifying the cost of i/o requests
CN109189578B (en) * 2018-09-06 2022-04-12 北京京东尚科信息技术有限公司 Storage server allocation method, device, management server and storage system
CN109358816B (en) * 2018-10-19 2022-05-20 网宿科技股份有限公司 Flow control method and device of distributed storage system
CN109800204B (en) * 2018-12-27 2021-03-05 深圳云天励飞技术有限公司 Data distribution method and related product
CN111177154B (en) * 2019-12-27 2023-07-25 掌迅亿通(北京)信息科技有限公司 Distributed database caching method and hash ring optimization thereof
CN111444040B (en) * 2020-01-20 2022-05-27 新华三大数据技术有限公司 Metadata backup method and device
US11379269B2 (en) * 2020-08-26 2022-07-05 International Business Machines Corporation Load balancing based on utilization percentage of CPU cores
CN113553179A (en) * 2021-07-16 2021-10-26 北京东方国信科技股份有限公司 Distributed key value storage load balancing method and system
CN116185643A (en) * 2023-04-23 2023-05-30 南京芯驰半导体科技有限公司 Load balancing method, device and equipment for hardware resources and storage medium
CN120295746A (en) * 2024-01-10 2025-07-11 杭州阿里云飞天信息技术有限公司 Multi-copy data processing method, device and electronic device

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
"基于对象存储系统的动态负载均衡算法". 覃灵军,冯丹,曾令仿,刘群.计算机科学,第Vol.33卷第No.5期. 2006 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102541460A (en) * 2010-12-20 2012-07-04 中国移动通信集团公司 Multiple disc management method and equipment
CN102541460B (en) * 2010-12-20 2014-10-08 中国移动通信集团公司 Multiple disc management method and equipment

Also Published As

Publication number Publication date
CN101013387A (en) 2007-08-08

Similar Documents

Publication Publication Date Title
CN100476742C (en) Load Balancing Method Based on Object Storage Device
CN111381928B (en) Virtual machine migration method, cloud computing management platform and storage medium
CN105653591B (en) A kind of industrial real-time data classification storage and moving method
CN106161120B (en) A distributed metadata management method for dynamic load balancing
CN103139302B (en) Real-time copy scheduling method considering load balancing
CN107302561B (en) A kind of hot spot data Replica placement method in cloud storage system
US20210255994A1 (en) Intelligent file system with transparent storage tiering
CN111857592B (en) Data storage method and device based on object storage system and electronic equipment
CN110417903B (en) A cloud computing-based information processing method and system
CN113835616B (en) Application data management method, system and computer device
CN110134495A (en) A container cross-host online migration method, storage medium and terminal device
CN110096350B (en) Cold and hot area division energy-saving storage method based on cluster node load state prediction
CN101763437A (en) Method and device for realizing high-speed buffer storage
CN106095531B (en) A virtual machine scheduling method based on grade and physical machine load in cloud platform
CN113608876B (en) Distributed file system metadata load balancing method based on load type awareness
CN115079958B (en) Multi-node load balancing cold and hot data migration device, method, terminal and medium
CN106610903A (en) Tiered storage system, storage controller, and method for deduplication and storage tiering
CN106527995A (en) Data expansion and migration method for I/O equilibrium
CN108932150A (en) Caching method, device and medium based on SSD and disk mixing storage
CN118502679B (en) Data access scheduling method and device for memory
CN108009196A (en) Database Rebalance Method
US11416152B2 (en) Information processing device, information processing method, computer-readable storage medium, and information processing system
WO2025176121A1 (en) Data processing
CN107689876A (en) The distribution management method of metadata in distributed objects storage system
US20150381520A1 (en) Data set multiplicity change device, server, data set multiplicity change method and computer redable medium

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20090408

Termination date: 20220209

CF01 Termination of patent right due to non-payment of annual fee