Decommissioning nodes in a hadoop cluster

How to decommission nodes/blacklist nodes

HDFS

Put following config in conf/hdfs-site.xml:

  dfs.hosts.exclude
  /full/path/of/host/exclude/file

Use following command to ask HDFS to re-read host exclude file and decommission nodes accordingly.
./bin/hadoop dfsadmin -refreshNodes

MapReduce

Put following config in conf/mapred-site.xml

  mapred.hosts.exclude
  /full/path/of/host/exclude/file

Use following command to ask Hadoop MapReduce to reconfigure nodes.
./bin/hadoop mradmin -refreshNodes

Whitelist/Recommission

Also you can "whitelist" nodes. In other words, you can specify which nodes are allowed to connect to namenode/jobtracker. 

HDFS

Put following config in conf/hdfs-site.xml:

  dfs.hosts
  /full/path/to/whitelisted/node/file

Use following command to ask Hadoop to refresh node status to based on configuration.
./bin/hadoop dfsadmin -refreshNodes

MapReduce

Put following config in conf/mapred-site.xml

  mapred.hosts
  >/full/path/to/whitelisted/node/file

Use following command to ask Hadoop MapReduce to reconfigure nodes.
./bin/hadoop mradmin -refreshNodes

Support of -mradmin was added in 0.21.0. See JIRA issue https://issues.apache.org/jira/browse/HADOOP-5643 for details.