CN108829738A - Date storage method and device in a kind of ceph - Google Patents
Date storage method and device in a kind of ceph Download PDFInfo
- Publication number
- CN108829738A CN108829738A CN201810499960.5A CN201810499960A CN108829738A CN 108829738 A CN108829738 A CN 108829738A CN 201810499960 A CN201810499960 A CN 201810499960A CN 108829738 A CN108829738 A CN 108829738A
- Authority
- CN
- China
- Prior art keywords
- storage
- domain
- subdomain
- stored
- virtual faults
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/004—Error avoidance
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention provides date storage method, device, electronic equipment and storage medium in a kind of ceph, method includes：Obtain the topological structure of memory device set group；Each physical fault domain of memory device set group is divided into multiple virtual faults subdomains；For each storage service to stored copies data, the first quantity based on the copy data for needing to store, select the first quantity virtual faults subdomain as storage failure domain, different virtual faults subdomains in selected storage failure domain belong to different physical fault domains, and different for the selected virtual faults subdomain of different storage services；By each storage service to stored copies data, it is stored respectively in the storage equipment of the corresponding each storage failure domain of the storage service.When carrying out data storage using date storage method provided in an embodiment of the present invention, some storage device fails does not interfere with multiple storage services.
Description
Technical field
The present invention relates to technical field of data administration, more particularly to number in a kind of ceph (a kind of distributed file system)
According to storage method, device, electronic equipment and storage medium.
Background technique
Ceph is a kind of distributed file system, for being stored, being managed to cloud storage data.
For ceph, in order to avoid loss of data, the mode for generalling use more copies stores data, and for more
A identical copy data needs to be stored in different physical fault domains, wherein physical fault domain is the storage artificially divided
Region, dividing physical fault domain is when breaking down in order to avoid some storage region, to be stored in the identical copies number in the region
According to can all be affected, therefore, multiple identical copy datas are respectively stored in different physical fault domains.
In the prior art, the physical fault domain range usually divided is excessive, it is understood that there may be some in physical fault domain is deposited
The case where storage equipment is stored with the data in multiple pool (data storage pool), wherein pool is the logical partition in ceph, is led to
A normal pool corresponds to a storage service.Therefore, using this storage method in the prior art, in physical fault domain
When one storage device fails, multiple storage services may be influenced whether.
Summary of the invention
The embodiment of the present invention is designed to provide date storage method and device in a kind of ceph, in this way into
When row data store, some storage device fails does not interfere with multiple storage services.
Specific technical solution is as follows：
The embodiment of the invention provides date storage method in a kind of ceph, the method includes：
The topological structure of memory device set group is obtained, the memory device set group includes multiple storage equipment, the topology
The dividing condition in the physical fault domain of memory device set group described in representation；
Each physical fault domain of the memory device set group is divided into multiple virtual faults subdomains；
For each storage service to stored copies data, based on the first quantity of the copy data for needing to store, choosing
The first quantity virtual faults subdomain is selected as storage failure domain, the selected different virtual faults stored in failure domain
Subdomain belongs to different physical fault domains, and different for the selected virtual faults subdomain of different storage services；
By each storage service to stored copies data, it is stored respectively in the corresponding each storage of the storage service
In the storage equipment of failure domain.
Optionally, it selects the first quantity virtual faults subdomain as before storing failure domain described, further includes：
Obtained virtual faults subdomain is divided into multiple groups virtual faults subdomain, and the difference in every group of virtual faults subdomain
Virtual faults subdomain belongs to different physical fault domains；
It is described to select the first quantity virtual faults subdomain as storage failure domain, including：
From one group of virtual faults subdomain in the multiple groups virtual faults subdomain, the virtual event of first quantity is selected
Hedge domain is as storage failure domain.
Optionally, the method also includes：
Based on the number of services of the newly-increased new storage service for needing stored copies data, by each physical fault domain not
The storage equipment of stored copies data is divided into number of services height storage domain；
For each new storage service to stored copies data, the second quantity based on the copy data that needs store,
The second quantity height storage domain is selected to store domain as dilatation, the difference son storage domain in selected dilatation storage domain belongs to
In different physical fault domains, and it is different for the different new selected sub- storage domains of storage service；
By each new storage service to stored copies data, it is corresponding each to be stored respectively in the new storage service
Dilatation stores in the storage equipment in domain.
Optionally, it is described select the second quantity height storage domain as dilatation storage domain before, further include：
Obtained son storage domain is divided into multiple groups storage domain, and the difference son storage domain in every group of son storage domain belongs to
Different physical fault domains；
It is described to select the second quantity height storage domain as dilatation storage domain, including：
From one group of son storage domain in multiple groups storage domain, select the second quantity height storage domain as expansion
Store storage domain.
It optionally, include multiple OSD in the storage equipment, and the copy data is stored in the storage equipment
OSD in.
The embodiment of the invention provides data storage device in a kind of ceph, described device includes：
Topological structure obtains module, and for obtaining the topological structure of memory device set group, the memory device set group includes
Multiple storage equipment, the topological structure indicate the dividing condition in the physical fault domain of the memory device set group；
Failure subdomain division module, for each physical fault domain of the memory device set group to be divided into multiple void
Quasi- failure subdomain；
Failure subdomain selecting module, for, to stored copies data, being stored based on needs for each storage service
First quantity of copy data selects the first quantity virtual faults subdomain as storage failure domain, selected storage
Different virtual faults subdomains in failure domain belong to different physical fault domains, and for the selected virtual event of different storage services
Hedge domain is different；
First memory module, for each storage service to stored copies data, to be stored respectively in the storage
In the storage equipment of the corresponding each storage failure domain of business.
Optionally, described device further includes：
First grouping module, the virtual faults subdomain for will obtain is divided into multiple groups virtual faults subdomain, and every group empty
Different virtual faults subdomains in quasi- failure subdomain belong to different physical fault domains；
The failure subdomain selecting module, is specifically used for：
From one group of virtual faults subdomain in the multiple groups virtual faults subdomain, the virtual event of first quantity is selected
Hedge domain is as storage failure domain.
Optionally, described device further includes：
Son storage domain division module, for the business number based on the newly-increased new storage service for needing stored copies data
The storage equipment of not stored copy data in each physical fault domain is divided into the number of services height and stores domain by amount；
Son storage domain selecting module, for for each new storage service to stored copies data, based on needing to store
Copy data the second quantity, select the second quantity height storage domain as dilatation storage domain, selected dilatation is deposited
Difference son storage domain in storage domain belongs to different physical fault domains, and equal for the different new selected sub- storage domains of storage service
It is different；
Second memory module, for stored copies data, it is new to be stored respectively in this for each new storage service
In the storage equipment in the corresponding each dilatation storage domain of storage service.
Optionally, described device further includes：
Second packet module, the son storage domain for that will obtain are divided into multiple groups storage domain, and in every group of son storage domain
Difference son storage domains belong to different physical fault domains；
The sub- storage domain selecting module, is specifically used for：
From one group of son storage domain in multiple groups storage domain, select the second quantity height storage domain as expansion
Store storage domain.
It optionally, include multiple OSD in the storage equipment, and the copy data is stored in the storage equipment
OSD in.
The embodiment of the invention also provides a kind of electronic equipment, including processor, communication interface, memory and communication are total
Line, wherein the processor, the communication interface, the memory complete mutual communication by the communication bus；
The memory, for storing computer program；
The processor when for executing the program stored on the memory, realizes any of the above-described method and step.
The embodiment of the invention also provides a kind of computer readable storage medium, the computer readable storage medium memory
Computer program is contained, the computer program realizes any of the above-described method and step when being executed by processor.
Use date storage method, device, electronic equipment and storage medium in ceph provided in an embodiment of the present invention, energy
It is enough that each physical fault domain of memory device set group is divided into multiple virtual faults subdomains, then it is directed to each storage service,
Virtual faults subdomain identical with copy amount is selected, and is for the selected virtual faults subdomain of each storage service
Different, then the copy data of each storage service is stored in the corresponding virtual faults subdomain of the storage service, thus
It can guarantee that the copy data of different storage services is stored in different virtual faults subdomains, therefore, when some virtual event
In hedge domain some storage device fails when, do not interfere with multiple storage services.
Certainly, it implements any of the products of the present invention or method must be not necessarily required to reach all the above excellent simultaneously
Point.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described.
Fig. 1 is a kind of flow chart of date storage method in ceph provided in an embodiment of the present invention；
Fig. 2 is a kind of schematic diagram of date storage method in ceph provided in an embodiment of the present invention；
Fig. 3 is another schematic diagram of date storage method in ceph provided in an embodiment of the present invention；
Fig. 4 is another schematic diagram of date storage method in ceph provided in an embodiment of the present invention；
Fig. 5 is a kind of structural schematic diagram of data storage device in ceph provided in an embodiment of the present invention；
Fig. 6 is a kind of structural schematic diagram of electronic equipment provided in an embodiment of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention is described.
The embodiment of the invention provides date storage method, device, electronic equipment and storage medium in a kind of ceph, energy
When enough solutions carry out data storage in the prior art, a storage device fails may influence whether multiple storage services
The technical issues of.
Referring to Fig. 1, Fig. 1 is a kind of flow chart of date storage method in ceph provided in an embodiment of the present invention, be can wrap
Include following steps：
Step S101：The topological structure of memory device set group is obtained, memory device set group includes multiple storage equipment, topology
The dividing condition in the physical fault domain of representation memory device set group；
In ceph, data copy can be stored in storage equipment, wherein storage equipment, which can be, has storage function
Electronic equipment or disk of energy etc., do not limit this.Before storing to data, storage equipment can be first obtained
The topological structure of cluster can obtain opening up for memory device set group by recalling crush map (middle layer ceph grade distribution map)
Structure is flutterred, wherein crush map describes the hierarchical structure of ceph storage system, such as the storage system includes how many rack,
Include how many storage equipment in each rack, contain in each storage equipment how many OSD (object storage device, it is right
As storing equipment) etc., wherein OSD is a kind of hardware store unit of entity.
In embodiments of the present invention, the topological structure of acquired memory device set group may include drawing for physical fault domain
Divide situation, wherein physical fault domain is the storage region artificially divided, and dividing physical fault domain is in order to avoid some memory block
When domain is broken down, the identical copies data for being stored in the region are all affected, therefore usually store identical copies data
In different physical fault domains.
Step S102：Each physical fault domain of memory device set group is divided into multiple virtual faults subdomains；
In this step, physics can be modified by way of modifying ceph rule (data storage rule in ceph)
The dividing condition of failure domain, wherein ceph rule defines the restrictive condition of copy data storage, for example, for needing to store
Three identical copy datas, ceph rule defines three copy datas which kind of rule to be stored in three differences with
Physical fault domain.
It in embodiments of the present invention, can be by modifying ceph rule for each physical fault of memory device set group
Domain is divided into multiple virtual faults subdomains.
Step S103：For each storage service to stored copies data, based on the copy data for needing to store the
One quantity, selects the first quantity virtual faults subdomain as storage failure domain, and the selected difference stored in failure domain is empty
Quasi- failure subdomain belongs to different physical fault domains, and different for the selected virtual faults subdomain of different storage services；
Step S104：By each storage service to stored copies data, it is corresponding every to be stored respectively in the storage service
In the storage equipment of a storage failure domain.
In embodiments of the present invention, can be according to the quantity of copy data after the completion of physical fault domain being divided, choosing should
The storage of copy amount virtual faults subdomain progress copy data.For example, the copy amount of some storage service is three, then may be used
Copy data is stored respectively with selecting three virtual faults subdomains, and selected three virtual faults subdomains belong to
Different physical fault domains can guarantee three identical copy datas being stored respectively in three different physical faults in this way
In domain.
In embodiments of the present invention, for each different storage service, selected virtual faults subdomain is different.It lifts
For example, Fig. 2 may refer to, Fig. 2 is a kind of schematic diagram of date storage method in ceph provided in an embodiment of the present invention, Fig. 2
Shown in embodiment, copy amount three, the number in physical fault domain is also three, as shown in Fig. 2, each rack is one
Physical fault domain, each physical fault domain are divided into three virtual faults subdomains, contain 3 in each virtual faults subdomain
A storage equipment.Then in embodiments of the present invention, for different storage services, can choose different virtual faults subdomains into
The storage of row data.
In embodiments of the present invention, it is assumed that there are three storage services, are denoted as business 1, business 2, business 3, to each respectively
When copy data in business is stored, different virtual faults subdomains can be first chosen for it.For example, can choose a1,
B1, c1 carry out the copy data in storage service 1, select a2, and b3, c3 carry out the copy data in storage service 2, select a3, b2, c2
Carry out the copy data in storage service 3, the virtual faults subdomain selected can be denoted as storage failure domain, for copy data
Storage.
Belong to different physical fault domains for virtual faults subdomain selected by each business, guarantees each industry in this way
Identical copy data in business is stored in different physical fault domains.For different business, selected virtual faults
Domain be it is different, the copy data of business each in this way is stored in respective virtual faults subdomain, between each business
It is independent of each other, if some storage device fails in some virtual faults subdomain, also only will affect virtual faults
The corresponding business in domain, will not influence other business.
As it can be seen that using date storage method in ceph provided in an embodiment of the present invention, it can be by the every of memory device set group
A physical fault domain is divided into multiple virtual faults subdomains, then is directed to each storage service, selects identical as copy amount
Virtual faults subdomain, and be different for the selected virtual faults subdomain of each storage service, then by each storage
The copy data of business is stored in the corresponding virtual faults subdomain of the storage service, ensures that different storage services in this way
Copy data be stored in different virtual faults subdomains, therefore, when some storage in some virtual faults subdomain is set
For when breaking down, multiple storage services are not interfered with.
In embodiments of the present invention, the first quantity virtual faults subdomain is being selected may be used also as before storing failure domain
To include：
Obtained virtual faults subdomain is divided into multiple groups virtual faults subdomain, and the difference in every group of virtual faults subdomain
Virtual faults subdomain belongs to different physical fault domains；
Select the first quantity virtual faults subdomain as storage failure domain, including：
From one group of virtual faults subdomain in multiple groups virtual faults subdomain, the first quantity virtual faults subdomain is selected to make
To store failure domain.
It is illustrated below with reference to Fig. 3, referring to Fig. 3, Fig. 3 is date storage method in ceph provided in an embodiment of the present invention
Another schematic diagram, in embodiment shown in Fig. 3, each rack is a physical fault domain, share 4 physical fault domains,
Each physical fault domain can be divided into multiple virtual faults subdomains by respectively rack A, rack B, rack C, rack D, can
With A1 as shown in Figure 3, A2, A3, B1, B2, B3, C1, C2, C3, D1, D2, D3.
Obtained virtual faults subdomain can be grouped, and every group virtual after the completion of physical fault domain divides
Different virtual faults subdomains in failure subdomain belong to different physical fault domains.For example, can be by A1, B1, C1, D1 points are one group
Virtual faults subdomain, by A2, B3, C3 points are one group of virtual faults subdomain, and by A3, B2, C2, D3 points are another group of virtual faults
The number in domain, the virtual faults subdomain for including in every group can be more than or equal to copy amount.
After determining grouping, for each storage service, one group can be selected to be used for from multiple groups virtual faults subdomain
Stored copies data, since the quantity for the failure domain for including in one group of virtual faults subdomain being selected may be greater than number of copies
Therefore amount can select copy amount virtual faults subdomain, by the business in the one group of virtual faults subdomain selected
Copy data be stored respectively in the copy amount virtual faults subdomain.For example, for business 1, if having selected A3, B2,
This group of virtual faults subdomain of C2, D3, and the business copy amount is three, then can select from this group of virtual faults subdomain
A3, B2, C2, the storage for copy data.
For each storage service, selected virtual faults subdomain is different, it is ensured that the pair of different storage services
Notebook data is stored in different virtual faults subdomains, therefore, when some storage equipment hair in some virtual faults subdomain
When raw failure, multiple storage services are not interfered with.
In embodiments of the present invention, when there is newly-increased storage service, stored copies data can be needed based on newly-increased
New storage service number of services, the storage equipment of not stored copy data in each physical fault domain is divided into business
Quantity height stores domain；
For each new storage service to stored copies data, the second quantity based on the copy data that needs store,
The second quantity height storage domain is selected to store domain as dilatation, the difference son storage domain in selected dilatation storage domain belongs to not
With physics failure domain, and it is different for the different new selected sub- storage domains of storage service；
By each new storage service to stored copies data, it is stored respectively in the corresponding each dilatation of the new storage service
In the storage equipment for storing domain.
It can be seen that being illustrated referring to fig. 4, Fig. 4 is the another kind of date storage method in ceph provided in an embodiment of the present invention
Schematic diagram, in embodiment shown in Fig. 4, each rack is a physical fault domain, shares 3 physical fault domains, respectively machine
Frame A, rack B, rack C, wherein virtual faults subdomain A1, A2, B1, B2, C1, C2 is used in existing storage service, if
Two storage services are newly increased on the basis of this, then can be based on the pair for each new storage service to stored copies data
The copy amount of notebook data selects quantity height storage domain as dilatation and stores domain.
May refer to Fig. 4, in Fig. 4 the storage equipment of not stored copy data in each physical fault domain divide in order to
Two son storage domains, respectively a1, a2, b1, b2, c1, c2, d1, d2 can be selected therefrom then for the storage service newly increased
Copy amount height storage domain is selected, sub- storage domain selected for each storage service belongs to different physical fault domains, right
It is different in the selected sub- storage domain of different storage services.Copy data to be stored can be then stored in selected
In son storage domain.
In embodiments of the present invention, it before selecting the second quantity height storage domain to store domain as dilatation, can also wrap
It includes：
Obtained son storage domain is divided into multiple groups storage domain, and the difference son storage domain in every group of son storage domain belongs to
Different physical fault domains；
The second quantity height storage domain is selected to store domain as dilatation, including：
From one group of son storage domain in multiple groups storage domain, the second quantity height storage domain is selected to store as dilatation
Domain.
Selecting storage service due to the sub process for storing domain of storage service selection for newly increasing and above is empty
The process of quasi- failure subdomain is essentially identical, may refer to above-described embodiment, this will not be repeated here.
As it can be seen that storage service is newly increased if necessary using date storage method in ceph provided in an embodiment of the present invention,
The storage equipment of not stored copy data in the ceph storage system can be divided into sub- storage domain, for each new storage industry
Business selects copy amount height storage domain to be used for the storage of copy data, and equal for the selected sub- storage domain of different business
Not identical, therefore, the storage service newly increased is also independent of each other, if some storage device fails, also only influences whether one
A storage service.Since the storage service newly increased does not occupy the storage equipment that original storage service has used, no
It will cause Data Migration, that is, do not need to redistribute data, would not also influence original storage service.
In embodiments of the present invention, copy data can be stored in the OSD in each storage equipment, each storage is set
Standby can include multiple OSD, and wherein OSD is a kind of hardware store unit of entity.
Referring to Fig. 5, Fig. 5 is a kind of structural schematic diagram of data storage device in ceph provided in an embodiment of the present invention, can
To include：
Topological structure obtains module 501, and for obtaining the topological structure of memory device set group, memory device set group includes more
A storage equipment, topological structure indicate the dividing condition in the physical fault domain of memory device set group；
Failure subdomain division module 502, for each physical fault domain of memory device set group to be divided into multiple void
Quasi- failure subdomain；
Failure subdomain selecting module 503, for for each storage service to stored copies data, based on needing to store
Copy data the first quantity, select the first quantity virtual faults subdomain as storage failure domain, selected storage is former
Different virtual faults subdomains in barrier domain belong to different physical fault domains, and are directed to the selected virtual faults of different storage services
Subdomain is different；
First memory module 504, for each storage service to stored copies data, to be stored respectively in the storage industry
It is engaged in the storage equipment of corresponding each storage failure domain.
In the embodiment of the present invention, in ceph shown in Fig. 5 on the basis of data storage device, can also include：
First grouping module, the virtual faults subdomain for will obtain is divided into multiple groups virtual faults subdomain, and every group empty
Different virtual faults subdomains in quasi- failure subdomain belong to different physical fault domains；
Failure subdomain selecting module, specifically can be used for：
From one group of virtual faults subdomain in multiple groups virtual faults subdomain, the first quantity virtual faults subdomain is selected to make
To store failure domain.
In embodiments of the present invention, can also include on the basis of data storage device in ceph shown in Fig. 5：
Son storage domain division module, for the business number based on the newly-increased new storage service for needing stored copies data
The storage equipment of not stored copy data in each physical fault domain is divided into number of services height storage domain by amount；
Son storage domain selecting module, for for each new storage service to stored copies data, based on needing to store
Copy data the second quantity, select the second quantity height storage domain to store domain, selected dilatation storage domain as dilatation
In difference son storage domains belong to different physical fault domains, and not for the different new selected sub- storage domains of storage service
Together；
Second memory module, for each new storage service to stored copies data, to be stored respectively in the new storage
In the storage equipment in the corresponding each dilatation storage domain of business.
Second packet module, the son storage domain for that will obtain are divided into multiple groups storage domain, and in every group of son storage domain
Difference son storage domains belong to different physical fault domains；
Son storage selecting module, specifically can be used for：
From one group of son storage domain in multiple groups storage domain, the second quantity height storage domain is selected to store as dilatation
Domain.
In embodiments of the present invention, storing includes multiple OSD in equipment, and copy data is stored in storage equipment
In OSD.
As it can be seen that using data storage device in ceph provided in an embodiment of the present invention, it can be by the every of memory device set group
A physical fault domain is divided into multiple virtual faults subdomains, then is directed to each storage service, selects identical as copy amount
Virtual faults subdomain, and be different for the selected virtual faults subdomain of each storage service, then by each storage
The copy data of business is stored in the corresponding virtual faults subdomain of the storage service, ensures that different storage services in this way
Copy data be stored in different virtual faults subdomains, therefore, when some storage in some virtual faults subdomain is set
For when breaking down, multiple storage services are not interfered with.
The embodiment of the invention discloses a kind of electronic equipment, as shown in Figure 6.Including processor 601, communication interface 602, deposit
Reservoir 603 and communication bus 604, wherein processor 601, communication interface 602, memory 603 are completed by communication bus 604
Mutual communication,
Memory 603, for storing computer program；
Processor 601 when for executing the program stored on memory 603, realizes any of the above-described method and step.
The communication bus that above-mentioned electronic equipment is mentioned can be Peripheral Component Interconnect standard (Peripheral Component
Interconnect, PCI) bus or expanding the industrial standard structure (Extended Industry Standard
Architecture, EISA) bus etc..The communication bus can be divided into address bus, data/address bus, control bus etc..For just
It is only indicated with a thick line in expression, figure, it is not intended that an only bus or a type of bus.
Communication interface is for the communication between above-mentioned electronic equipment and other equipment.
Memory may include random access memory (Random Access Memory, RAM), also may include non-easy
The property lost memory (Non-Volatile Memory, NVM), for example, at least a magnetic disk storage.Optionally, memory may be used also
To be storage device that at least one is located remotely from aforementioned processor.
Above-mentioned processor can be general processor, including central processing unit (Central Processing Unit,
CPU), network processing unit (Network Processor, NP) etc.；It can also be digital signal processor (Digital Signal
Processing, DSP), it is specific integrated circuit (Application Specific Integrated Circuit, ASIC), existing
It is field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic device, discrete
Door or transistor logic, discrete hardware components.
The embodiment of the invention also provides a kind of computer readable storage medium, it is stored in computer readable storage medium
Computer program realizes any of the above-described method and step when computer program is executed by processor.
In the above-described embodiments, can come wholly or partly by software, hardware, firmware or any combination thereof real
It is existing.When implemented in software, it can entirely or partly realize in the form of a computer program product.Computer program product
Including one or more computer instructions.When loading on computers and executing computer program instructions, all or part of real estate
Raw process or function according to the embodiment of the present invention.Computer can be general purpose computer, special purpose computer, computer network,
Or other programmable devices.Computer instruction may be stored in a computer readable storage medium, or from a computer
Readable storage medium storing program for executing to another computer readable storage medium transmit, for example, computer instruction can from a web-site,
Computer, server or data center by wired (such as coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wireless (such as
Infrared, wireless, microwave etc.) mode transmitted to another web-site, computer, server or data center.Computer
Readable storage medium storing program for executing can be any usable medium or include one or more usable medium collection that computer can access
At the data storage devices such as server, data center.Usable medium can be magnetic medium, (for example, floppy disk, hard disk, magnetic
Band), optical medium (for example, DVD) or semiconductor medium (such as solid state hard disk Solid State Disk (SSD)) etc..
It should be noted that, in this document, relational terms such as first and second and the like are used merely to a reality
Body or operation are distinguished with another entity or operation, are deposited without necessarily requiring or implying between these entities or operation
In any actual relationship or order or sequence.Moreover, the terms "include", "comprise" or its any other variant are intended to
Non-exclusive inclusion, so that the process, method, article or equipment including a series of elements is not only wanted including those
Element, but also including other elements that are not explicitly listed, or further include for this process, method, article or equipment
Intrinsic element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that
There is also other identical elements in process, method, article or equipment including element.
Each embodiment in this specification is all made of relevant mode and describes, same and similar portion between each embodiment
Dividing may refer to each other, and each embodiment focuses on the differences from other embodiments.Especially for device,
For electronic equipment, computer readable storage medium and computer program product embodiments, since it is substantially similar to method
Embodiment, so being described relatively simple, the relevent part can refer to the partial explaination of embodiments of method.
The above is merely preferred embodiments of the present invention, it is not intended to limit the scope of the present invention.It is all in this hair
Any modification, equivalent replacement, improvement and so within bright spirit and principle, are included within the scope of protection of the present invention.
Claims (12)
1. date storage method in a kind of distributed file system ceph, which is characterized in that the method includes：
The topological structure of memory device set group is obtained, the memory device set group includes multiple storage equipment, the topological structure
Indicate the dividing condition in the physical fault domain of the memory device set group；
Each physical fault domain of the memory device set group is divided into multiple virtual faults subdomains；
Institute is selected based on the first quantity of the copy data for needing to store to stored copies data for each storage service
The first quantity virtual faults subdomain is stated as storage failure domain, the selected different virtual faults subdomains stored in failure domain
Belong to different physical fault domains, and different for the selected virtual faults subdomain of different storage services；
By each storage service to stored copies data, it is stored respectively in the corresponding each storage failure of the storage service
In the storage equipment in domain.
2. the method according to claim 1, wherein in selection the first quantity virtual faults subdomain
Before storage failure domain, further include：
Obtained virtual faults subdomain is divided into multiple groups virtual faults subdomain, and the difference in every group of virtual faults subdomain is virtual
Failure subdomain belongs to different physical fault domains；
It is described to select the first quantity virtual faults subdomain as storage failure domain, including：
From one group of virtual faults subdomain in the multiple groups virtual faults subdomain, the first quantity virtual faults is selected
Domain is as storage failure domain.
3. the method according to claim 1, wherein the method also includes：
It, will be not stored in each physical fault domain based on the number of services of the newly-increased new storage service for needing stored copies data
The storage equipment of copy data is divided into number of services height storage domain；
For each new storage service to stored copies data, based on the second quantity of the copy data for needing to store, selection
The second quantity height storage domain stores domain as dilatation, and the difference son storage domain in selected dilatation storage domain belongs to not
With physics failure domain, and it is different for the different new selected sub- storage domains of storage service；
By each new storage service to stored copies data, it is stored respectively in the corresponding each dilatation of the new storage service
In the storage equipment for storing domain.
4. according to the method described in claim 3, it is characterized in that, it is described select the second quantity height storage domain as
Before dilatation stores domain, further include：
Obtained son storage domain is divided into multiple groups storage domain, and the difference son storage domain in every group of son storage domain belongs to difference
Physical fault domain；
It is described to select the second quantity height storage domain as dilatation storage domain, including：
From one group of son storage domain in multiple groups storage domain, the second quantity height storage domain is selected to deposit as dilatation
Store up domain.
5. method described in -4 according to claim 1, which is characterized in that include multiple object-storage units in the storage equipment
OSD, and the copy data is stored in the OSD in the storage equipment.
6. data storage device in a kind of distributed file system ceph, which is characterized in that described device includes：
Topological structure obtains module, and for obtaining the topological structure of memory device set group, the memory device set group includes multiple
Equipment is stored, the topological structure indicates the dividing condition in the physical fault domain of the memory device set group；
Failure subdomain division module, for each physical fault domain of the memory device set group to be divided into multiple virtual events
Hedge domain；
Failure subdomain selecting module, for for each storage service to stored copies data, the copy stored based on needs
First quantity of data selects the first quantity virtual faults subdomain as storage failure domain, selected storage failure
Different virtual faults subdomains in domain belong to different physical fault domains, and for selected virtual faults of different storage services
Domain is different；
First memory module, for each storage service to stored copies data, to be stored respectively in the storage service
In the storage equipment of corresponding each storage failure domain.
7. device according to claim 6, which is characterized in that described device further includes：
First grouping module, the virtual faults subdomain for will obtain are divided into multiple groups virtual faults subdomain, and every group of virtually event
Different virtual faults subdomains in hedge domain belong to different physical fault domains；
The failure subdomain selecting module, is specifically used for：
From one group of virtual faults subdomain in the multiple groups virtual faults subdomain, the first quantity virtual faults is selected
Domain is as storage failure domain.
8. device according to claim 6, which is characterized in that described device further includes：
Son storage domain division module will for the number of services based on the newly-increased new storage service for needing stored copies data
The storage equipment of not stored copy data is divided into number of services height storage domain in each physical fault domain；
Son storage domain selecting module, for for each new storage service to stored copies data, the pair stored based on needs
Second quantity of notebook data selects the second quantity height storage domain as dilatation and stores domain, and selected dilatation stores domain
In difference son storage domains belong to different physical fault domains, and not for the different new selected sub- storage domains of storage service
Together；
Second memory module, for each new storage service to stored copies data, to be stored respectively in the new storage
In the storage equipment in the corresponding each dilatation storage domain of business.
9. device according to claim 8, which is characterized in that described device further includes：
Second packet module, the son storage domain for that will obtain are divided into multiple groups storage domain, and in every group of son storage domain not
Belong to different physical fault domains with son storage domain；
The sub- storage domain selecting module, is specifically used for：
From one group of son storage domain in multiple groups storage domain, the second quantity height storage domain is selected to deposit as dilatation
Store up domain.
10. according to device described in claim 6-9, which is characterized in that include that the storage of multiple objects is single in the storage equipment
First OSD, and the copy data is stored in the OSD in the storage equipment.
11. a kind of electronic equipment, which is characterized in that including processor, communication interface, memory and communication bus, wherein described
Processor, the communication interface, the memory complete mutual communication by the communication bus；
The memory, for storing computer program；
The processor when for executing the program stored on the memory, realizes any side claim 1-5
Method step.
12. a kind of computer readable storage medium, which is characterized in that be stored with computer in the computer readable storage medium
Program realizes claim 1-5 any method and step when the computer program is executed by processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810499960.5A CN108829738B (en) | 2018-05-23 | 2018-05-23 | Data storage method and device in ceph |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810499960.5A CN108829738B (en) | 2018-05-23 | 2018-05-23 | Data storage method and device in ceph |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108829738A true CN108829738A (en) | 2018-11-16 |
| CN108829738B CN108829738B (en) | 2020-12-25 |
Family
ID=64148366
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810499960.5A Active CN108829738B (en) | 2018-05-23 | 2018-05-23 | Data storage method and device in ceph |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108829738B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111628893A (en) * | 2020-05-27 | 2020-09-04 | 星辰天合（北京）数据科技有限公司 | Fault processing method and device of distributed storage system and electronic equipment |
| CN111857735A (en) * | 2020-07-23 | 2020-10-30 | 浪潮云信息技术股份公司 | A method and system for Crush creation based on Rook deployment Ceph |
| CN113742309A (en) * | 2021-08-18 | 2021-12-03 | Oppo广东移动通信有限公司 | Copy storage method, device, storage medium and computer equipment |
| CN114466030A (en) * | 2021-12-27 | 2022-05-10 | 天翼云科技有限公司 | Management method and device for data distribution storage strategy and distribution storage system |
| CN115080204A (en) * | 2022-05-13 | 2022-09-20 | 曙光信息产业股份有限公司 | Virtual machine scheduling method and device, electronic equipment and storage medium |
| CN116112512A (en) * | 2022-12-28 | 2023-05-12 | 中国人寿保险股份有限公司上海数据中心 | A Distributed Storage System Based on Fault Domain |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070156763A1 (en) * | 2005-12-30 | 2007-07-05 | Jian-Hong Liu | Storage management system and method thereof |
| US20090276654A1 (en) * | 2008-05-02 | 2009-11-05 | International Business Machines Corporation | Systems and methods for implementing fault tolerant data processing services |
| CN105706056A (en) * | 2013-10-03 | 2016-06-22 | 微软技术许可有限责任公司 | Fault domains on modern hardware |
| CN107704212A (en) * | 2017-10-31 | 2018-02-16 | 紫光华山信息技术有限公司 | A kind of data processing method and device |
-
2018
- 2018-05-23 CN CN201810499960.5A patent/CN108829738B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070156763A1 (en) * | 2005-12-30 | 2007-07-05 | Jian-Hong Liu | Storage management system and method thereof |
| US20090276654A1 (en) * | 2008-05-02 | 2009-11-05 | International Business Machines Corporation | Systems and methods for implementing fault tolerant data processing services |
| CN105706056A (en) * | 2013-10-03 | 2016-06-22 | 微软技术许可有限责任公司 | Fault domains on modern hardware |
| CN107704212A (en) * | 2017-10-31 | 2018-02-16 | 紫光华山信息技术有限公司 | A kind of data processing method and device |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111628893A (en) * | 2020-05-27 | 2020-09-04 | 星辰天合（北京）数据科技有限公司 | Fault processing method and device of distributed storage system and electronic equipment |
| CN111628893B (en) * | 2020-05-27 | 2022-07-12 | 北京星辰天合科技股份有限公司 | Fault handling method and device for distributed storage system, and electronic equipment |
| CN111857735A (en) * | 2020-07-23 | 2020-10-30 | 浪潮云信息技术股份公司 | A method and system for Crush creation based on Rook deployment Ceph |
| CN113742309A (en) * | 2021-08-18 | 2021-12-03 | Oppo广东移动通信有限公司 | Copy storage method, device, storage medium and computer equipment |
| CN113742309B (en) * | 2021-08-18 | 2024-11-08 | Oppo广东移动通信有限公司 | Copy storage method, device, storage medium and computer equipment |
| CN114466030A (en) * | 2021-12-27 | 2022-05-10 | 天翼云科技有限公司 | Management method and device for data distribution storage strategy and distribution storage system |
| CN114466030B (en) * | 2021-12-27 | 2024-03-12 | 天翼云科技有限公司 | Management method and device of data distributed storage strategy and distributed storage system |
| CN115080204A (en) * | 2022-05-13 | 2022-09-20 | 曙光信息产业股份有限公司 | Virtual machine scheduling method and device, electronic equipment and storage medium |
| CN116112512A (en) * | 2022-12-28 | 2023-05-12 | 中国人寿保险股份有限公司上海数据中心 | A Distributed Storage System Based on Fault Domain |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108829738B (en) | 2020-12-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108829738A (en) | Date storage method and device in a kind of ceph | |
| CN103929454B (en) | The method and system of load balancing storage in a kind of cloud computing platform | |
| US11409453B2 (en) | Storage capacity forecasting for storage systems in an active tier of a storage environment | |
| CN110737401B (en) | Method, apparatus and computer program product for managing redundant array of independent disks | |
| CN105242879B (en) | A data storage method and protocol server | |
| US11520512B2 (en) | Method for storage management, electronic device and computer program product | |
| JP6211631B2 (en) | Identifying workloads and sizing buffers for volume replication purposes | |
| CN109873714B (en) | Cloud computing node configuration updating method and terminal equipment | |
| CN103001793B (en) | Method and device for managing ACL (access control list) | |
| CN108804568B (en) | Method and device for storing copy data in Openstack in ceph | |
| CN113126890A (en) | Method, apparatus and computer program product for storage | |
| CN105490843A (en) | Information processing method and system | |
| CN104378394B (en) | The update method and device of a kind of server cluster files | |
| US11320988B2 (en) | Method, apparatus and computer program product for managing disk array | |
| CN103345519B (en) | Method and device for data distribution of shared-nothing distributed database | |
| CN109963316B (en) | Multipath routing method and device for mobile satellite network | |
| CN108846009A (en) | Copy data storage method and device in a kind of ceph | |
| CN112748848A (en) | Method, apparatus and computer program product for storage management | |
| US20140157221A1 (en) | Techniques for routing signal wires in an integrated circuit design | |
| US20240422006A1 (en) | Simplified Hash Table | |
| CN112748849A (en) | Method, apparatus and computer program product for storing data | |
| CN107273316A (en) | Parallel FLASH accesses system and method | |
| CN116016028B (en) | Method, system, medium and equipment for creating multiple network interfaces for Pod based on IPvlan | |
| CN117724651A (en) | Data distribution method, device and equipment | |
| CN110058790B (en) | Method, apparatus and computer program product for storing data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |