<PRoperty> <name>dfs.hosts.exclude</name> <value>file_path</value> </property> 2、file_path文件中存儲要離線的幾點名稱3、執行命令 hdfs dfsadmin -refreshNodes 問題:執行到最發現50070界面數據塊不發生變化,查看namenode發現問題:2017-02-08 15:19:10,145 WARN org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy: Failed to place enough replicas, still in need of 1 to reach 2 (unavailableStorages=[DISK, ARCHIVE], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=false) All required storage types are unavailable: unavailableStorages=[DISK, ARCHIVE], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]} 原因:namenode中記錄的文件副本數和實際存儲的副本數不一致,可以使用 hadoop fsck / >test.log命令檢查解決方法: hdfs hadoop dfs -setrep -w 2 -R / 將機器中文件的副本數統一 然后再執行hadoop fsck / >test1.log檢查,看是否有Missing4、然后重新執行hdfs dfsadmin -refreshNodes5、退役完成(block 轉移結束),后續hadoop會自動刪除datanode上的數據,也可以直接停掉datanode,手動刪除數據下線tasktracker or nodemanager(過程與下線datanode類似,一下列舉不同點)
如下配置項到mapred-site.xml
[html] view plain copy<property> <name>mapred.hosts.exclude</name> <value>mrhosts.exclude</value> </property> yarn rmadmin -refreshNodes 若沒啟用yarn,即下線tasktracker時執行:hadoop mradmin -refreshNodes
新聞熱點
疑難解答