Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
17/09/29 10:03:49 ERROR Shell: Failed to locate the winutils binary in the hadoop binary path
java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:356)
at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:371)
at org.apache.hadoop.util.Shell.<clinit>(Shell.java:364)
at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:80)
..........................................................................................................................
經(jīng)常在調(diào)試spark的時(shí)候出現(xiàn)上面的異常情況疚宇，其原因是windows系統(tǒng)下環(huán)境變量的未配置問題冲粤，解決這個(gè)問題的方案：
- 下載一個(gè)winutils.exe楣富，http://public-repo-1.hortonworks.com/hdp-win-alpha/winutils.exe占卧。
- 配置HADOOP_HOME的環(huán)境變量涣觉，復(fù)制winutils.exe 到${HADOOP_HOME}/bin目錄下
- 替代方案贵涵，在程序中設(shè)置System.setProperty("hadoop.home.dir", "hadoop的home目錄");
