[go: up one dir, main page]

Li et al., 2018 - Google Patents

Mist: Efficient dissemination of erasure-coded data in data centers

Li et al., 2018

View PDF
Document ID
9596447087699338720
Author
Li J
Li B
Li B
Publication year
Publication venue
IEEE Transactions on Emerging Topics in Computing

External Links

Snippet

Data centers store a massive amount of data in a large number of servers built by commodity hardware. To maintain data integrity against server failures, erasure codes have been extensively deployed in modern data centers to provide a higher level of failure tolerance …
Continue reading at iqua.ece.toronto.edu (PDF) (other versions)

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Error detection; Error correction; Monitoring responding to the occurence of a fault, e.g. fault tolerance
    • G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
    • G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
    • G06F11/1076—Parity data used in redundant arrays of independent storages, e.g. in RAID systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Error detection; Error correction; Monitoring responding to the occurence of a fault, e.g. fault tolerance
    • G06F11/16—Error detection or correction of the data by redundancy in hardware
    • G06F11/20—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F15/00—Digital computers in general; Data processing equipment in general
    • G06F15/16—Combinations of two or more digital computers each having at least an arithmetic unit, a programme unit and a register, e.g. for a simultaneous processing of several programmes
    • G06F15/163—Interprocessor communication
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30—Information retrieval; Database structures therefor; File system structures therefor
    • G06F17/30067—File systems; File servers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30—Information retrieval; Database structures therefor; File system structures therefor
    • G06F17/30286—Information retrieval; Database structures therefor; File system structures therefor in structured data stores
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30—Information retrieval; Database structures therefor; File system structures therefor
    • G06F17/30861—Retrieval from the Internet, e.g. browsers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for programme control, e.g. control unit
    • G06F9/06—Arrangements for programme control, e.g. control unit using stored programme, i.e. using internal store of processing equipment to receive and retain programme
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F2211/00—Indexing scheme relating to details of data-processing equipment not covered by groups G06F3/00 - G06F13/00
    • G06F2211/10—Indexing scheme relating to G06F11/10
    • G06F2211/1002—Indexing scheme relating to G06F11/1076
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network-specific arrangements or communication protocols supporting networked applications
    • H04L67/10—Network-specific arrangements or communication protocols supporting networked applications in which an application is distributed across nodes in the network
    • H04L67/1097—Network-specific arrangements or communication protocols supporting networked applications in which an application is distributed across nodes in the network for distributed storage of data in a network, e.g. network file system [NFS], transport mechanisms for storage area networks [SAN] or network attached storage [NAS]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring

Similar Documents

Publication Publication Date Title
US10241695B2 (en) Optimizing rebuilds when using multiple information dispersal algorithms
US9672110B1 (en) Transmission time refinement in a storage system
CN103810061B (en) A kind of High Availabitity cloud storage method
CN103209210B (en) Method for improving erasure code based storage cluster recovery performance
EP3005786A1 (en) Adjusting dispersed storage network traffic due to rebuilding
CN103984607A (en) Distributed storage method, device and system
WO2016130091A1 (en) Methods of encoding and storing multiple versions of data, method of decoding encoded multiple versions of data and distributed storage system
US10437673B2 (en) Internet based shared memory in a distributed computing system
Zhu et al. On the speedup of recovery in large-scale erasure-coded storage systems
US9037679B2 (en) Efficient high availability storage systems
US10296633B1 (en) Data storage management system
Lee et al. Erasure coded storage systems for cloud storage—challenges and opportunities
US10152248B2 (en) Erasure coding for elastic cloud storage
Li et al. Relieving both storage and recovery burdens in big data clusters with R-STAIR codes
Li et al. Mist: Efficient dissemination of erasure-coded data in data centers
Cheng et al. Toward Load-Balanced Redundancy Transitioning for Erasure-Coded Storage
Zhang et al. SA-RSR: a read-optimal data recovery strategy for XOR-coded distributed storage systems
US10254980B1 (en) Scheduling requests from data sources for efficient data decoding
US10334045B2 (en) Indicating multiple encoding schemes in a dispersed storage network
Mohan et al. Geo-aware erasure coding for high-performance erasure-coded storage clusters
US20180089022A1 (en) Combined slice objects in alternate memory locations
Shi et al. EC-Bench: Benchmarking Onload and Offload Erasure Coders on Modern Hardware Architectures
Wu et al. A high-speed Cauchy CODEC algorithm for distributed storage system
Kim et al. Accelerating erasure coding by exploiting multiple repair paths in distributed storage systems
KR102744003B1 (en) Erasure Recovery in Distributed Storage Systems