設(shè)計(jì)目標(biāo)及假設(shè)
1蛤铜、Hardware Failure
2、Streaming Data Access (適用于批處理場(chǎng)景丛肢,不適用于交互場(chǎng)景)
3围肥、Large Data Sets
4、Simple Coherency Model (數(shù)據(jù)不可修改)
5蜂怎、Moving Computation is Cheaper than Moving Data
整體架構(gòu)
基本組件
1虐先、NameNode: maintains the file system namespace. Any change to the file system namespace or its properties is recorded by the NameNode. An application can specify the number of replicas of a file that should be maintained by HDFS. The number of copies of a file is called the replication factor of that file.
2、DataNode:存儲(chǔ)對(duì)應(yīng)的數(shù)據(jù)派敷,支持一次寫蛹批,多次讀,不能修改篮愉。
3腐芍、Client:對(duì)應(yīng)的客戶端
存儲(chǔ)策略
相關(guān)shell命令
HDFS Quotas Guide
Name Quotas
hdfs dfsadmin -setQuota N <directory> <directory>
hdfs dfsadmin -clrQuota N <directory> <directory>
Space Quotas
hdfs dfsadmin -setSpaceQuota N <directory> <directory>
hdfs dfsadmin -clrSpaceQuota N <directory> <directory>
Storage Type Quotas
hdfs dfsadmin -setSpaceQuota ?-storageType <directory> <directory>
hdfs dfsadmin -clrSpaceQuota-storageType ?