hadoop distcp <source> <target>
hadoop distcp <source> <source> <source> <source> <target>

Confirm you can list the source and target namenodes.

hdfs dfs -ls hdfs://node1:8020
hdfs dfs -ls hdfs://node2:8020

Copy from remote(node1) to local

hadoop distcp hdfs://node1:8020/user/root/test_data distcp_target

Copy from local to remote(node2)

hadoop distcp test_data hdfs://node2:8020/user/root/distcp_target

Older HDP Versions

If you are copying from an older HDP cluster use hftp.
NOTE: Port and protocol are slightly different.

hadoop distcp hftp://node1:50070/user/root/test/sample.txt distcp_target
  • bigdata/distcp.txt
  • Last modified: 2018/05/25 14:05
  • (external edit)