image
useradd hadoop
/etc/hadoop/hadoop-env.sh
JAVA_HOME=${JAVA_HOME}
JAVA_HOME=/usr/local/java
配置文件
公共參數(shù)
core-site.xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
-->
<!-- Put site-specific property overrides in this file. -->
<configuration>
<!-- name node-->
<property>
<name>fs.defaultFS</name>
<value>hdfs://mini1:9000</value>
</property>
<!-- 進(jìn)程工作的數(shù)據(jù)目錄-->
<property>
<name>hadoop.tmp.dir</name>
<value>/home/hadoop/hdpdata</value>
</property>
</configuration>
hdfs配置（可直接使用默認(rèn)值）
hdfs-site.xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
-->
<!-- Put site-specific property overrides in this file. -->
<configuration>
<!-- 副本的數(shù)量，默認(rèn)值 3 -->
<property>
<name>dfs.replication</name>
<value>2</value>
</property>
</configuration>
mapreduce配置
mapred-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
-->
<!-- Put site-specific property overrides in this file. -->
<configuration>
<!-- 運(yùn)行平臺 默認(rèn)local -->
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
yarn-site.xml
<?xml version="1.0"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
-->
<configuration>
<!-- Site specific YARN configuration properties -->
<property>
<name>yarn.resourcemanager.hostname</name>
<value>mini1</value>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
</configuration>
hadoop添加環(huán)境變量
sudo vim /etc/profile
# HADOOP
export HADOOP_HOME=/home/hadoop/apps/hadoop-2.8.5
export PATH=$PATH:${HADOOP_HOME}/bin:${HADOOP_HOME}/sbin
hdfs格式化
hadoop namenode -format
image
啟動hadoop
namenode
hadoop-daemon start namenode
hadoop-daemon stop namenode
http://****:50070
(http://10.0.1.101:50070/dfshealth.html#tab-overview)
hadoop-daemon start datanode
hadoop-daemon stop datanode
批量啟動
配置節(jié)點(diǎn)
vi etc/hadoop/slaves
mini2
mini3
mini4
執(zhí)行命令
start-all.sh(不推薦)
start-hdfs.sh
start-yarn.sh
問題
[hadoop@mini1 hadoop]$ ssh-copy-id mini1
- /usr/bin/ssh-copy-id: ERROR: No identities found
~/.ssh/id_rsa id_rsa.pub文件不存在
執(zhí)行ssh-keygen 解決 - WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
