這個是 因為 在 yarn 執(zhí)行MapReduce的使用 找不到 hadoop 的classpath
一般在 shell中輸入
hadoop classpath
輸出為
/etc/hadoop/conf:/opt/cloudera/parcels/CDH-5.14.2-1.cdh5.14.2.p0.3/lib/hadoop/libexec/../../hadoop/lib/*:/opt/cloudera/parcels/CDH-5.14.2-1.cdh5.14.2.p0.3/lib/hadoop/libexec/../../hadoop/.//*:/opt/cloudera/parcels/CDH-5.14.2-1.cdh5.14.2.p0.3/lib/hadoop/libexec/../../hadoop-hdfs/./:/opt/cloudera/parcels/CDH-5.14.2-1.cdh5.14.2.p0.3/lib/hadoop/libexec/../../hadoop-hdfs/lib/*:/opt/cloudera/parcels/CDH-5.14.2-1.cdh5.14.2.p0.3/lib/hadoop/libexec/../../hadoop-hdfs/.//*:/opt/cloudera/parcels/CDH-5.14.2-1.cdh5.14.2.p0.3/lib/hadoop/libexec/../../hadoop-yarn/lib/*:/opt/cloudera/parcels/CDH-5.14.2-1.cdh5.14.2.p0.3/lib/hadoop/libexec/../../hadoop-yarn/.//*:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/*:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//*
然后 這些輸出 直接 粘貼到 yarn 的配置文件里 yarn-site.xml
的 屬性 <name>yarn.application.classpath</name>
賦值
對于 使用cdh 安裝的 hadoop
直接修改 /etc/hadoop/conf/yarn-site.xml 然后重啟yarn
yarn nodemanager &
yarn resourcemanager &
還有這個錯誤
Container launch failed for container_1531560184035_0010_01_000003 : org.apache.hadoop.yarn.exceptions.InvalidAuxServiceException: The auxService:mapreduce_shuffle does not exist
也是修改 yarn-site.xml 后重啟
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce_shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
檢驗 MapReduce 是否可用
hadoop jar ./hadoop-mapreduce-examples-2.8.2.jar wordcount /derby.log /outputs
