| 一种副本复制和纠错码融合的云存储文件系统容错机制 |
杨东日1,王颖1( ),刘鹏2 |
| 2. 中国人民解放军理工大学, 南京 210007 |
| Fault-tolerant mechanism combined with replication and error correcting code for cloud file systems |
| Dongri YANG1,Ying WANG1(),Peng LIU2 |
| 1. College of Engineering & Information Technology,University of Chinese Academy of Sciences, Beijing 100049, China 2. PLA University of Science and Technology, Nanjing 210007, China |
摘要:
| |||
| 文章导读 | |||
| 摘要在云存储技术中,云存储文件系统的数据容错十分重要,直接关系到整个系统的可用性。该文通过对有中心的分布式文件系统进行分析,提出了双机热备的元数据管理节点容错技术和块副本与基于纠删编解码算法相结合的存储节点容错技术,为云存储文件系统设计了双重保险的容错机制。实验结果表明,该机制大大提高了云存储系统的可靠性,并提高了磁盘空间利用率。 | |||
| 关键词 ：云存储,容错,双机热备,副本复制,纠删码 | |||
| Abstract：Fault tolerant is important for the reliability of cloud storage file systems. This paper analyzes the reliability of typical cloud file systems with a central metadata server and proposes a fault-tolerant mechanism that combines replication schemes with error correcting codes for storage node reliability guarantee as well as a hot-standby scheme for the metadata server reliability guarantee. Experimental results demonstrate that the mechanism improves the reliability of current cloud storage file systems and at the same time improves the storage utilizations compared with replication schemes. | |||
| Key words：cloud storagefault toleranthot stand-byreplicationerasure codes | |||
| 收稿日期: 2013-09-25 出版日期: 2015-04-16 | |||
| |||
| 基金资助: | |||
| 引用本文: |
| 杨东日, 王颖, 刘鹏. 一种副本复制和纠错码融合的云存储文件系统容错机制[J]. 清华大学学报（自然科学版）, 2014, 54(1): 137-144. Dongri YANG, Ying WANG, Peng LIU. Fault-tolerant mechanism combined with replication and error correcting code for cloud file systems. Journal of Tsinghua University(Science and Technology), 2014, 54(1): 137-144. |
| 链接本文: |
| http://jst.tsinghuajournals.com/CN/或 http://jst.tsinghuajournals.com/CN/Y2014/V54/I1/137 |
图表:
 |
| 容错系统的模块设计 |
 |
| 主备元数据同步流程图 |
 |
| 编解码数据块分布式存储示意图 |
 |
| 数据写入过程示意图 |
 |
| 数据读出过程示意图 |
 |
| 采用基于RS编解码算法的容错技术的云存储站点的可靠性 |
 |
| 云存储站点的可靠性与编码冗余倍数及文件分块数的关系图 |
参考文献:
| [1] | Wang Y, Yang D R, Li P. CloStor: A cloud storage system for fast large-scale data I/O [M]//Advance in Computer Science and Its Applications. Springer Berlin Heidelberg, 2014: 1023-1030. |
| [2] | Ghemawat S, Gobioff H, Leung S T. The Google file system [C]// Proc of the Symp on Operating Systems Principles (SOSP 2003). Bolton: ACM Press, 2003: 29-43. |
| [3] | Shvachko K, Kuang H, Radia S, et al. The Hadoop distributed file system [C]// Proc of the IEEE 26th Symp on MSST. Lake Tahoe: IEEE, 2010: 1-10. |
| [4] | Decandia G, Hastorun D, Jampani M, et al.Dynamo: Amazon's highly available key-value store [C]// Proc of the SOSP 2007. Stevenson: ACM Press, 2007: 205-220. |
| [5] | Lakshman A, Malik P. Cassandra: A decentralized structured storage system [J]. ACM SIGOPS Operating Systems Review, 2010, 44(2): 35-40. |
| [6] | Bhagwat D, Pollack K, Long D D E, et al. Providing high reliability in a minimum redundancy archival storage system [C]// Proc of the 14th IEEE International Symposium on MASCOTS. 2006: 413-421. |
| [7] | Spillers N. Storage challenges in the medical industry [C]// The 4th Intelligent Storage Workshop. Digital Technology Center, University of Minnesota, 2006. |
| [8] | Wicker S B, Bhargava V K. Reed-Solomon Codes and Their Applications [M]. Piscataway, NJ: IEEE Press, 1983. |
| [9] | Luby M G, Mitzenmacher M, Shokrollahi M A, et al.Efficient erasure correcting codes[J]. IEEE Transactions on Information Theory, 2001, 47(2): 569-584. |
相关文章:
|
