正文之前

畢業(yè)答辯的時(shí)候，答辯老師說我的最大的問題就是沒有和大數(shù)據(jù)扯上關(guān)系券敌。所以后面就要想方設(shè)法扯點(diǎn)關(guān)系上去唾戚。被逼無奈的我。只能繼續(xù)開始琢磨Hadoop了待诅。以前其實(shí)我是想做的叹坦。。但是ubuntu上的Hadoop一度坑的我想死卑雁。募书。所以就沒有繼續(xù)了。测蹲∮瘢。現(xiàn)在只能重操舊業(yè)了弛房。道盏。

正文

1：安裝Homebrew

這個(gè)是Mac上安裝軟件的神器。文捶。荷逞。賊雞兒好用。粹排。种远。誰用誰知道！

Homebrew的官方網(wǎng)站

安裝Homebrew的方法顽耳，命令行鍵入下面的命令：

 /usr/bin/ruby -e "$(curl -fsSLhttps://raw.githubusercontent.com/Homebrew/install/master/install)"

ps:homebrew常用shell命

• 查看brew的幫助brew -help
• 安裝軟件brew install hadoop
• 卸載軟件 brew uninstall hadoop
• 搜索軟件 brew search hadoop
• 查看已經(jīng)安裝的軟件brew list
  *更新軟件 brew update
• 更新某具體軟件 brew upgrade hadoop

2:ssh登錄本地

首先在系統(tǒng)里打開遠(yuǎn)程登錄：

1:首先生成ssh公鑰坠敷，終端命令代碼如下

ssh-keygen -t rsa -P ""

cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

如果提示缺少文件的話妙同，那就創(chuàng)建個(gè)文件好了。不過應(yīng)該不會(huì)膝迎。粥帚。我是先創(chuàng)建然后再操作的。限次。

2:接下來進(jìn)行測(cè)試登錄本地是否成功芒涡，在 Terminal 里輸入以下代碼查看能不能免密 ssh 到 localhost：

ssh localhost

3:登錄成功顯示結(jié)果如下：

3:安裝Hadoop

3.1 : 輸入以下代碼，自動(dòng)安裝hadoop:

brew install hadoop

ps：通過Homebrew安裝軟件后卖漫，軟件目錄一般位于/usr/local/Cellar,并軟件目錄里面帶有版本號(hào). 如圖我的hadoop安裝目錄如下：

4:測(cè)試Hadoop是否安裝成功

Hadoop有三種安裝模式：?jiǎn)螜C(jī)模式费尽，偽分布式模式，分布式模式 分布式模式需要在多臺(tái)電腦上面測(cè)試羊始，這里只測(cè)試 偽分布式模式

4.1:測(cè)試偽分布式模式

測(cè)試為分布模式前旱幼，需要修改相關(guān)的5個(gè)配置文件，把homebrew默認(rèn)的單機(jī)模式修改成偽分布式模式

可以使用nano或者vim進(jìn)行配置文件的修改突委，我這里使用sublime文本編輯器修改柏卤，方便省事兒。另外就是在sublime里面經(jīng)常沒法找到那些隱藏文件夾下的文件鸯两。所以這里我提供一個(gè)好辦法闷旧，用命令行啟動(dòng)sublime！逼格滿滿還好用钧唐！

sublime 提供命令行的工具叫subs忙灼，路徑在：

/Applications/Sublime\ Text.app/Contents/SharedSupport/bin/subl

使用ln命令
ln命令是做文件鏈接用的，不恰當(dāng)?shù)睦涌梢苑Q之為建立快捷方式钝侠。

ln  /Applications/Sublime\ Text.app/Contents/SharedSupport/bin/subl /usr/local/bin/subl

如上命令是將sublime提供的subl命令鏈接到usr/local/bin/subl 這個(gè)路徑上來该园，usr/local/bin 這個(gè)文件夾是全局都可以訪問到的命令所在的目錄，都是我們自己下載安裝的一些命令行工具鏈接所在地 比如npm帅韧、httpie等

搞完了之后里初，簡(jiǎn)單的這樣就ok！

下面進(jìn)行文件修改忽舟，修改路徑為：

你按照你的來就好

修改core-site.xml（位置 etc/hadoop/）双妨，改參數(shù)如下：

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
    <property>
        <name>hadoop.tmp.dir</name>
        <value>/usr/local/Cellar/hadoop/hdfs/tmp</value>
        <description>A base for other temporary directories</description>
    </property>
    <property>
        <name>fs.default.name</name>
        <value>hdfs://localhost:9000</value>
    </property>
</configuration>

.修改mapred-site.xml （位置 etc/hadoop/），改參數(shù)如下：

如果文件后綴是 .xml.example叮阅，改為 .xml刁品。

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
    <property>
        <name>mapred.job.tracker</name>
        <value>localhost:9010</value>
    </property>
    <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>
</configuration>

變量mapred.job.tracker 保存了JobTracker的位置，因?yàn)橹挥蠱apReduce組件需要知道這個(gè)位置浩姥，所以它出現(xiàn)在mapred-site.xml文件中挑随。

修改hdfs-site.xml（位置 etc/hadoop/），改參數(shù)如下：

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
</configuration>

變量dfs.replication指定了每個(gè)HDFS數(shù)據(jù)庫的復(fù)制次數(shù)勒叠。 通常為3, 由于我們只有一臺(tái)主機(jī)和一個(gè)偽分布式模式的DataNode兜挨，將此值修改為1膏孟。

修改yarn-site.xml（位置 etc/hadoop/），改參數(shù)如下：

<?xml version="1.0"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->
<configuration>

<!-- Site specific YARN configuration properties -->

    <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>

</configuration>

修改hadoop-env.sh（位置 etc/hadoop/）拌汇，改參數(shù)如下：然后開啟hadoop-env.sh里的注釋

#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Set Hadoop-specific environment variables here.

##
## THIS FILE ACTS AS THE MASTER FILE FOR ALL HADOOP PROJECTS.
## SETTINGS HERE WILL BE READ BY ALL HADOOP COMMANDS.  THEREFORE,
## ONE CAN USE THIS FILE TO SET YARN, HDFS, AND MAPREDUCE
## CONFIGURATION OPTIONS INSTEAD OF xxx-env.sh.
##
## Precedence rules:
##
## {yarn-env.sh|hdfs-env.sh} > hadoop-env.sh > hard-coded defaults
##
## {YARN_xyz|HDFS_xyz} > HADOOP_xyz > hard-coded defaults
##

# Many of the options here are built from the perspective that users
# may want to provide OVERWRITING values on the command line.
# For example:
#
#  JAVA_HOME=/usr/java/testing hdfs dfs -ls
#
# Therefore, the vast majority (BUT NOT ALL!) of these defaults
# are configured for substitution and not append.  If append
# is preferable, modify this file accordingly.

###
# Generic settings for HADOOP
###

# Technically, the only required environment variable is JAVA_HOME.
# All others are optional.  However, the defaults are probably not
# preferred.  Many sites configure these options outside of Hadoop,
# such as in /etc/profile.d

# The java implementation to use. By default, this environment
# variable is REQUIRED on ALL platforms except OS X!
export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0_172.jdk/Contents/Home

# Location of Hadoop.  By default, Hadoop will attempt to determine
# this location based upon its execution path.
# export HADOOP_HOME=

# Location of Hadoop's configuration information.  i.e., where this
# file is living. If this is not defined, Hadoop will attempt to
# locate it based upon its execution path.
#
# NOTE: It is recommend that this variable not be set here but in
# /etc/profile.d or equivalent.  Some options (such as
# --config) may react strangely otherwise.
#
# export HADOOP_CONF_DIR=${HADOOP_HOME}/etc/hadoop

# The maximum amount of heap to use (Java -Xmx).  If no unit
# is provided, it will be converted to MB.  Daemons will
# prefer any Xmx setting in their respective _OPT variable.
# There is no default; the JVM will autoscale based upon machine
# memory size.
# export HADOOP_HEAPSIZE_MAX=

# The minimum amount of heap to use (Java -Xms).  If no unit
# is provided, it will be converted to MB.  Daemons will
# prefer any Xms setting in their respective _OPT variable.
# There is no default; the JVM will autoscale based upon machine
# memory size.
# export HADOOP_HEAPSIZE_MIN=

# Enable extra debugging of Hadoop's JAAS binding, used to set up
# Kerberos security.
# export HADOOP_JAAS_DEBUG=true

# Extra Java runtime options for all Hadoop commands. We don't support
# IPv6 yet/still, so by default the preference is set to IPv4.
# export HADOOP_OPTS="-Djava.net.preferIPv4Stack=true"
# For Kerberos debugging, an extended option set logs more invormation
export HADOOP_OPTS="-Djava.net.preferIPv4Stack=true -Dsun.security.krb5.debug=true -Dsun.security.spnego.debug"

# Some parts of the shell code may do special things dependent upon
# the operating system.  We have to set this here. See the next
# section as to why....
export HADOOP_OS_TYPE=${HADOOP_OS_TYPE:-$(uname -s)}


# Under certain conditions, Java on OS X will throw SCDynamicStore errors
# in the system logs.
# See HADOOP-8719 for more information.  If one needs Kerberos
# support on OS X, one will want to change/remove this extra bit.
case ${HADOOP_OS_TYPE} in
  Darwin*)
    export HADOOP_OPTS="${HADOOP_OPTS} -Djava.security.krb5.realm= "
    export HADOOP_OPTS="${HADOOP_OPTS} -Djava.security.krb5.kdc= "
    export HADOOP_OPTS="${HADOOP_OPTS} -Djava.security.krb5.conf= "
  ;;
esac

# Extra Java runtime options for some Hadoop commands
# and clients (i.e., hdfs dfs -blah).  These get appended to HADOOP_OPTS for
# such commands.  In most cases, # this should be left empty and
# let users supply it on the command line.
# export HADOOP_CLIENT_OPTS=""

#
# A note about classpaths.
#
# By default, Apache Hadoop overrides Java's CLASSPATH
# environment variable.  It is configured such
# that it sarts out blank with new entries added after passing
# a series of checks (file/dir exists, not already listed aka
# de-deduplication).  During de-depulication, wildcards and/or
# directories are *NOT* expanded to keep it simple. Therefore,
# if the computed classpath has two specific mentions of
# awesome-methods-1.0.jar, only the first one added will be seen.
# If two directories are in the classpath that both contain
# awesome-methods-1.0.jar, then Java will pick up both versions.

# An additional, custom CLASSPATH. Site-wide configs should be
# handled via the shellprofile functionality, utilizing the
# hadoop_add_classpath function for greater control and much
# harder for apps/end-users to accidentally override.
# Similarly, end users should utilize ${HOME}/.hadooprc .
# This variable should ideally only be used as a short-cut,
# interactive way for temporary additions on the command line.
# export HADOOP_CLASSPATH="/some/cool/path/on/your/machine"

# Should HADOOP_CLASSPATH be first in the official CLASSPATH?
# export HADOOP_USER_CLASSPATH_FIRST="yes"

# If HADOOP_USE_CLIENT_CLASSLOADER is set, the classpath along
# with the main jar are handled by a separate isolated
# client classloader when 'hadoop jar', 'yarn jar', or 'mapred job'
# is utilized. If it is set, HADOOP_CLASSPATH and
# HADOOP_USER_CLASSPATH_FIRST are ignored.
# export HADOOP_USE_CLIENT_CLASSLOADER=true

# HADOOP_CLIENT_CLASSLOADER_SYSTEM_CLASSES overrides the default definition of
# system classes for the client classloader when HADOOP_USE_CLIENT_CLASSLOADER
# is enabled. Names ending in '.' (period) are treated as package names, and
# names starting with a '-' are treated as negative matches. For example,
# export HADOOP_CLIENT_CLASSLOADER_SYSTEM_CLASSES="-org.apache.hadoop.UserClass,java.,javax.,org.apache.hadoop."

# Enable optional, bundled Hadoop features
# This is a comma delimited list.  It may NOT be overridden via .hadooprc
# Entries may be added/removed as needed.
# export HADOOP_OPTIONAL_TOOLS="hadoop-openstack,hadoop-aliyun,hadoop-azure,hadoop-azure-datalake,hadoop-aws,hadoop-kafka"

###
# Options for remote shell connectivity
###

# There are some optional components of hadoop that allow for
# command and control of remote hosts.  For example,
# start-dfs.sh will attempt to bring up all NNs, DNS, etc.

# Options to pass to SSH when one of the "log into a host and
# start/stop daemons" scripts is executed
# export HADOOP_SSH_OPTS="-o BatchMode=yes -o StrictHostKeyChecking=no -o ConnectTimeout=10s"

# The built-in ssh handler will limit itself to 10 simultaneous connections.
# For pdsh users, this sets the fanout size ( -f )
# Change this to increase/decrease as necessary.
# export HADOOP_SSH_PARALLEL=10

# Filename which contains all of the hosts for any remote execution
# helper scripts # such as workers.sh, start-dfs.sh, etc.
# export HADOOP_WORKERS="${HADOOP_CONF_DIR}/workers"

###
# Options for all daemons
###
#

#
# Many options may also be specified as Java properties.  It is
# very common, and in many cases, desirable, to hard-set these
# in daemon _OPTS variables.  Where applicable, the appropriate
# Java property is also identified.  Note that many are re-used
# or set differently in certain contexts (e.g., secure vs
# non-secure)
#

# Where (primarily) daemon log files are stored.
# ${HADOOP_HOME}/logs by default.
# Java property: hadoop.log.dir
# export HADOOP_LOG_DIR=${HADOOP_HOME}/logs

# A string representing this instance of hadoop. $USER by default.
# This is used in writing log and pid files, so keep that in mind!
# Java property: hadoop.id.str
# export HADOOP_IDENT_STRING=$USER

# How many seconds to pause after stopping a daemon
# export HADOOP_STOP_TIMEOUT=5

# Where pid files are stored.  /tmp by default.
# export HADOOP_PID_DIR=/tmp

# Default log4j setting for interactive commands
# Java property: hadoop.root.logger
# export HADOOP_ROOT_LOGGER=INFO,console

# Default log4j setting for daemons spawned explicitly by
# --daemon option of hadoop, hdfs, mapred and yarn command.
# Java property: hadoop.root.logger
# export HADOOP_DAEMON_ROOT_LOGGER=INFO,RFA

# Default log level and output location for security-related messages.
# You will almost certainly want to change this on a per-daemon basis via
# the Java property (i.e., -Dhadoop.security.logger=foo). (Note that the
# defaults for the NN and 2NN override this by default.)
# Java property: hadoop.security.logger
# export HADOOP_SECURITY_LOGGER=INFO,NullAppender

# Default process priority level
# Note that sub-processes will also run at this level!
# export HADOOP_NICENESS=0

# Default name for the service level authorization file
# Java property: hadoop.policy.file
# export HADOOP_POLICYFILE="hadoop-policy.xml"

#
# NOTE: this is not used by default!  <-----
# You can define variables right here and then re-use them later on.
# For example, it is common to use the same garbage collection settings
# for all the daemons.  So one could define:
#
# export HADOOP_GC_SETTINGS="-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps"
#
# .. and then use it as per the b option under the namenode.

###
# Secure/privileged execution
###

#
# Out of the box, Hadoop uses jsvc from Apache Commons to launch daemons
# on privileged ports.  This functionality can be replaced by providing
# custom functions.  See hadoop-functions.sh for more information.
#

# The jsvc implementation to use. Jsvc is required to run secure datanodes
# that bind to privileged ports to provide authentication of data transfer
# protocol.  Jsvc is not required if SASL is configured for authentication of
# data transfer protocol using non-privileged ports.
# export JSVC_HOME=/usr/bin

#
# This directory contains pids for secure and privileged processes.
#export HADOOP_SECURE_PID_DIR=${HADOOP_PID_DIR}

#
# This directory contains the logs for secure and privileged processes.
# Java property: hadoop.log.dir
# export HADOOP_SECURE_LOG=${HADOOP_LOG_DIR}

#
# When running a secure daemon, the default value of HADOOP_IDENT_STRING
# ends up being a bit bogus.  Therefore, by default, the code will
# replace HADOOP_IDENT_STRING with HADOOP_xx_SECURE_USER.  If one wants
# to keep HADOOP_IDENT_STRING untouched, then uncomment this line.
# export HADOOP_SECURE_IDENT_PRESERVE="true"

###
# NameNode specific parameters
###

# Default log level and output location for file system related change
# messages. For non-namenode daemons, the Java property must be set in
# the appropriate _OPTS if one wants something other than INFO,NullAppender
# Java property: hdfs.audit.logger
# export HDFS_AUDIT_LOGGER=INFO,NullAppender

# Specify the JVM options to be used when starting the NameNode.
# These options will be appended to the options specified as HADOOP_OPTS
# and therefore may override any similar flags set in HADOOP_OPTS
#
# a) Set JMX options
# export HDFS_NAMENODE_OPTS="-Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.port=1026"
#
# b) Set garbage collection logs
# export HDFS_NAMENODE_OPTS="${HADOOP_GC_SETTINGS} -Xloggc:${HADOOP_LOG_DIR}/gc-rm.log-$(date +'%Y%m%d%H%M')"
#
# c) ... or set them directly
# export HDFS_NAMENODE_OPTS="-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -Xloggc:${HADOOP_LOG_DIR}/gc-rm.log-$(date +'%Y%m%d%H%M')"

# this is the default:
# export HDFS_NAMENODE_OPTS="-Dhadoop.security.logger=INFO,RFAS"

###
# SecondaryNameNode specific parameters
###
# Specify the JVM options to be used when starting the SecondaryNameNode.
# These options will be appended to the options specified as HADOOP_OPTS
# and therefore may override any similar flags set in HADOOP_OPTS
#
# This is the default:
# export HDFS_SECONDARYNAMENODE_OPTS="-Dhadoop.security.logger=INFO,RFAS"

###
# DataNode specific parameters
###
# Specify the JVM options to be used when starting the DataNode.
# These options will be appended to the options specified as HADOOP_OPTS
# and therefore may override any similar flags set in HADOOP_OPTS
#
# This is the default:
# export HDFS_DATANODE_OPTS="-Dhadoop.security.logger=ERROR,RFAS"

# On secure datanodes, user to run the datanode as after dropping privileges.
# This **MUST** be uncommented to enable secure HDFS if using privileged ports
# to provide authentication of data transfer protocol.  This **MUST NOT** be
# defined if SASL is configured for authentication of data transfer protocol
# using non-privileged ports.
# This will replace the hadoop.id.str Java property in secure mode.
# export HDFS_DATANODE_SECURE_USER=hdfs

# Supplemental options for secure datanodes
# By default, Hadoop uses jsvc which needs to know to launch a
# server jvm.
# export HDFS_DATANODE_SECURE_EXTRA_OPTS="-jvm server"

###
# NFS3 Gateway specific parameters
###
# Specify the JVM options to be used when starting the NFS3 Gateway.
# These options will be appended to the options specified as HADOOP_OPTS
# and therefore may override any similar flags set in HADOOP_OPTS
#
# export HDFS_NFS3_OPTS=""

# Specify the JVM options to be used when starting the Hadoop portmapper.
# These options will be appended to the options specified as HADOOP_OPTS
# and therefore may override any similar flags set in HADOOP_OPTS
#
# export HDFS_PORTMAP_OPTS="-Xmx512m"

# Supplemental options for priviliged gateways
# By default, Hadoop uses jsvc which needs to know to launch a
# server jvm.
# export HDFS_NFS3_SECURE_EXTRA_OPTS="-jvm server"

# On privileged gateways, user to run the gateway as after dropping privileges
# This will replace the hadoop.id.str Java property in secure mode.
# export HDFS_NFS3_SECURE_USER=nfsserver

###
# ZKFailoverController specific parameters
###
# Specify the JVM options to be used when starting the ZKFailoverController.
# These options will be appended to the options specified as HADOOP_OPTS
# and therefore may override any similar flags set in HADOOP_OPTS
#
# export HDFS_ZKFC_OPTS=""

###
# QuorumJournalNode specific parameters
###
# Specify the JVM options to be used when starting the QuorumJournalNode.
# These options will be appended to the options specified as HADOOP_OPTS
# and therefore may override any similar flags set in HADOOP_OPTS
#
# export HDFS_JOURNALNODE_OPTS=""

###
# HDFS Balancer specific parameters
###
# Specify the JVM options to be used when starting the HDFS Balancer.
# These options will be appended to the options specified as HADOOP_OPTS
# and therefore may override any similar flags set in HADOOP_OPTS
#
# export HDFS_BALANCER_OPTS=""

###
# HDFS Mover specific parameters
###
# Specify the JVM options to be used when starting the HDFS Mover.
# These options will be appended to the options specified as HADOOP_OPTS
# and therefore may override any similar flags set in HADOOP_OPTS
#
# export HDFS_MOVER_OPTS=""

###
# Router-based HDFS Federation specific parameters
# Specify the JVM options to be used when starting the RBF Routers.
# These options will be appended to the options specified as HADOOP_OPTS
# and therefore may override any similar flags set in HADOOP_OPTS
#
# export HDFS_DFSROUTER_OPTS=""
###

###
# Advanced Users Only!
###

#
# When building Hadoop, one can add the class paths to the commands
# via this special env var:
# export HADOOP_ENABLE_BUILD_PATHS="true"

#
# To prevent accidents, shell commands be (superficially) locked
# to only allow certain users to execute certain subcommands.
# It uses the format of (command)_(subcommand)_USER.
#
# For example, to limit who can execute the namenode command,
# export HDFS_NAMENODE_USER=hdfs

這里記得要改掉Java的位置捌馍！！５Ｃ汀幕垦！

export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0_172.jdk/Contents/Home

4.2：運(yùn)行hadoop以及查看遠(yuǎn)端

第一步：進(jìn)入文件夾（重要重要重要：后續(xù)所有操作一定要先進(jìn)入當(dāng)前hadoop文件夾）

cd /usr/local/Cellar/hadoop/3.0.0

第二步：格式化文件系統(tǒng)（是對(duì)namenode進(jìn)行初始化）：

./bin/hdfs namenode -format

啟動(dòng) NameNode 和 DataNode：

./sbin/start-dfs.sh

localhost:3.1.0 zhangzhaobo$ ./sbin/start-dfs.sh
Starting namenodes on [localhost]
Starting datanodes
Starting secondary namenodes [localhost]
2018-06-03 14:12:02,890 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
localhost:3.1.0 zhangzhaobo$ 

報(bào)錯(cuò)沒關(guān)系。傅联。隨它去~

現(xiàn)在可以直接看到Overview 界面了：

http://localhost:9870 Overview

讓 HDFS 可以被用來執(zhí)行 MapReduce jobs：

./bin/hdfs dfs -mkdir /user

./bin/hdfs dfs -mkdir /user/input

localhost:3.1.0 zhangzhaobo$ ./bin/hdfs dfs -mkdir /user
2018-06-03 14:13:45,029 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
localhost:3.1.0 zhangzhaobo$  ./bin/hdfs dfs -mkdir /user/input
2018-06-03 14:13:51,659 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
localhost:3.1.0 zhangzhaobo$ 

把input改成你想要命名的任意子文件夾名字即可，這里我選擇命名input.

啟動(dòng) ResourceManager 和 NodeManager：

./sbin/start-yarn.sh

localhost:3.1.0 zhangzhaobo$ ./sbin/start-yarn.sh
Starting resourcemanager
Starting nodemanagers
localhost:3.1.0 zhangzhaobo$ 

現(xiàn)在你就可以看到All Applications 界面了：

http://localhost:8088 All Applications

上面大部分內(nèi)容來自簡(jiǎn)書的兩位優(yōu)秀程序猿同學(xué)疚察。蒸走。。通報(bào)下：

macOS安裝hadoop 3.0.0偽分布式教程

在終端中使用sublime打開文件

OK貌嫡！下面進(jìn)入深化拓展１茸ぁ！

首先就是可能因?yàn)榘惭bJava的時(shí)候默認(rèn)安裝了一些版本岛抄。别惦。所以會(huì)有一些奇形怪狀的錯(cuò)誤出來。夫椭。這里介紹下如何查看版本以及改變版本：

首先輸入代碼看看你把 Java 裝到哪里了 ：

/usr/libexec/java_home

輸入代碼： java -version

如果已經(jīng)裝了Java掸掸，你會(huì)看到類似醬紫結(jié)果：

localhost:3.1.0 zhangzhaobo$ java -version
java version "1.8.0_172"
Java(TM) SE Runtime Environment (build 1.8.0_172-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.172-b11, mixed mode)

如果沒有 ，用 Homebrew 安裝了 Java8蹭秋，老鐵靖避， 求你一定要加那個(gè)8.剩晴。不然9的話別怪我沒警告你！

brew cask install java8

如果你本來就安裝了Java9 其實(shí)也沒關(guān)系，一起裝了就好了Ｒ恋琛！你看我的：

一起裝了之后犀变，你只要告訴系統(tǒng)茸歧，我要的默認(rèn)Java是哪個(gè)就ok了！這個(gè)時(shí)候輸入如下命令：

subl ~/.bash_profile

然后開始編輯

基本上按照我這個(gè)就沒啥錯(cuò)誤了丈挟。刁卜。

export JAVA_HOME_9=/Library/Java/JavaVirtualMachines/jdk-9.0.1.jdk/Contents/Home
export JAVA_HOME_8=/Library/Java/JavaVirtualMachines/jdk1.8.0_172.jdk/Contents/Home
export JAVA_HOME=$JAVA_HOME_8
alias jdk8='export JAVA_HOME=$JAVA_HOME_8'
alias jdk9='export JAVA_HOME=$JAVA_HOME_9'

然后你再去那個(gè)地方改一下：

沒錯(cuò)，就是這里＝负濉长酗！

修改hadoop-env.sh（位置 etc/hadoop/），改參數(shù)如下：然后開啟hadoop-env.sh里的注釋桐绒，JAVA_HOME那兒改成后來的這個(gè)就ok了夺脾！然后你就發(fā)現(xiàn)啥事沒有了之拨。。不然你在8088那個(gè)網(wǎng)址肯定出錯(cuò)咧叭。蚀乔。。

正文之后

溜了溜了菲茬。今晚健身完回去好好學(xué)習(xí)吉挣。。把ubuntu和Raspberry都搭個(gè)Hadoop婉弹，做個(gè)小集群睬魂。。镀赌。哇咔咔氯哮！畢設(shè)完成有望啊Ｉ谭稹：砀帧！

我覺得自己賊6.良姆。肠虽。在圖書館控制windows然后windows局域網(wǎng)內(nèi)VNC樹莓派。玛追。税课。三位一體。等ubuntu也弄個(gè)向日葵丟上去豹缀。那就太可怕了伯复！