hadoop屈梁、hbase、zookeeper是大數(shù)據(jù)里面三個(gè)重要的產(chǎn)品榛了，關(guān)于這三個(gè)產(chǎn)品的介紹在讶，各種書籍和網(wǎng)站介紹得很多，在這里不再贅述忽冻，僅介紹在ubunt下如何安裝和配置這三種產(chǎn)品真朗。
第一步：配置虛擬機(jī)為橋接模式
第二步：安裝openssh-server，并且登錄不需要
/home/jerry/>apt-get install openssh-server #安裝openssh-server
/home/jerry/>ssh localhost #登錄本機(jī)僧诚，需要密碼
/home/jerry/>exit# 退出剛才的 ssh localhost
/home/jerry/>cd ~/.ssh/# 若沒(méi)有該目錄遮婶，請(qǐng)先執(zhí)行一次ssh localhost
/home/jerry/>ssh-keygen -t rsa# 會(huì)有提示，都按回車就可以
/home/jerry/>cat ./id_rsa.pub >> ./authorized_keys # 加入授權(quán)
/home/jerry/>ssh localhost#不需要密碼
第三步：安裝JDK 1.7以上
/home/jerry/>tar -xzf jdk-8u161-linux-x64.tar.gz -C /usr/local
/home/jerry/>cd /usr/local
/usr/local/>mv jdk1.8.0_161/ java
/usr/local/>gedit ~/.bashrc
export JAVA_HOME=/usr/local/java
export JRE_HOME=$JAVA_HOME/jre
export CLASSPATH=.:$JAVA_HOME/lib:$JRE_HOME/lib
export PATH=$PATH:$JAVA_HOME/bin
/usr/local/>source ~/.bashrc
/usr/local/>java -version
出現(xiàn)版本信息湖笨，說(shuō)明配置正確旗扑。
第四步：安裝hadoop
/home/jerry/>tar -xzf hadoop-2.6.4.tar.gz -C /usr/local
/home/jerry/>cd /usr/local
/usr/local/>mv hadoop-2.6.4/ hadoop
/usr/local/>chown -R root ./hadoop
/usr/local/>gedit ~/.bashrc
export HADOOP_HOME=/usr/local/hadoop
export PATH=$PATH:$HADOOP_HOME/bin
export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib/native"
export JAVA_LIBRAY_PATH=$HADOOP_HOME/lib/native
export LD_LIBRARY_PATH=$JAVA_LIBRARY_PATH
/usr/local/>source ~/.bashrc
/usr/local/>hadoop version
#查看hsddop版本有內(nèi)容，說(shuō)明配置正確
/usr/local/>gedit /usr/local/hadoop/etc/hadoop/core-site.xml
<configuration>
<property>
<name>fs.defualtFS</name>
<value>hdfs://localhost:9000</value>
</property>
<configuration>
? ? <property>
? ? ? ? <name>hadoop.tmp.dir</name>
? ? ? ? <value>file:/usr/local/hadoop/tmp</value>
? ? ? ? <description>Abase for other temporary directories.</description>
? ? </property>
</configuration>
/usr/local/>gedit /usr/local/hadoop/etc/hadoop/hdfs-site.xml
<configuration>
? ? <property>
? ? ? ? <name>dfs.replication</name>
? ? ? ? <value>1</value>
? ? </property>
? ? <property>
? ? ? ? <name>dfs.namenode.name.dir</name>
? ? ? ? <value>file:/usr/local/hadoop/tmp/dfs/name</value>
? ? </property>
? ? <property>
? ? ? ? <name>dfs.datanode.data.dir</name>
? ? ? ? <value>file:/usr/local/hadoop/tmp/dfs/data</value>
? ? </property>
<property>
<name>dfs.namenode.http-address</name>
<value>0.0.0.0:50070</value>
</property>
</configuration>
dfs.namenode.name.dir：namenode位置
dfs.namenode.data.dir：datanode位置
dfs.namenode.http-address：hadoop WEB端口
gedit /usr/local/hadoop/etc/hadoop/hdoop-env.sh
將語(yǔ)句
export JAVA_HOME=$JAVA_HOME
修改為
export JAVA_HOME=/usr/java/jdk1.8.0_101
hadoop WEB 頁(yè)面：http://localhost:50070
hadoop+hbase+zookeeper安裝指南
/usr/local/>cd hadoop
/usr/local/hadoop/>hdfs namenode -format
/usr/local/hadoop/>sbin/start-dfs.sh
/usr/local/hadoop/>cp etc/hadoop/mapred-site.xml.template etc/hadoop/mapred-site.xml
/usr/local/hadoop/>gedit /usr/local/hadoop/etc/hadoop/yarn-site.xml
<configuration>
<property>
<name>yarn.nodemanager.aux-server</name>
<value>mapreduce_shuffle</value>
</property>
</configuration>
/usr/local/hadoop/>sbin/start-dfs.sh(./sbin/stop-dfs.sh)
/usr/local/hadoop/>sbin/start-yarn.sh(./sbin/stop-yarn.sh)
/usr/local/hadoop/>
sbin/mr-jobhistory-daemon.sh start historyserver #查看歷史任務(wù)( ./sbin/mr-jobhistory-daemon.sh stop historyserver)
hadoop安全模式慈省，比如hadoop非正常退出臀防，會(huì)進(jìn)入安全模式
$hdfs dfsadmin -safemode get|wait|entry|leave|wait
get：獲得當(dāng)前是否進(jìn)入安全模式
wait：等待進(jìn)入安全模式
entry：強(qiáng)制進(jìn)入安全模式
leave：強(qiáng)制離開(kāi)安全模式
第五步：安裝HBase
/home/jerry/>tar -xzf hbase-2.2.4-bin.tar.gz -C /usr/local
/home/jerry/>cd /usr/local
/usr/local/>mv hbase-2.2.4/ hbase
/usr/local/>gedit ~/.bashrc
export HBASE_HOME=/usr/local/hbase
export PATH=$PATH:$HBASE_HOME/bin
/usr/local/>source ~/.bashrc
/usr/local/>chown -R root ./habase
/usr/local/>gedit hbase/conf/hbase-site.xml
<configuration>
? ? <property>
<name>hbase.rootdir</name>
<value>hdfs://localhost:9000/hbase</value>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
? <property>
? ? ? <name>hbase.unsafe.stream.capability.enforce</name>
? ? ? <value>false</value>
? </property>
? <property>
? ? ? ? <name>hbase.zookeeper.property.dataDir</name>
? ? ? ? <value>/usr/local/zookeeper</value>
? ? </property>
</configuration>
hbase.rootdir：hbase在HDFS的位置，端口號(hào)一定要與/usr/local/hadoop/etc/hadoop/core-site.xml保持一致边败。
hbase.cluster.distributed：是否為分布式袱衷，在這里為偽分布式
hbase.unsafe.stream.capability.enforce：在分布式情況下, 一定設(shè)置為false（hbase的一個(gè)bug）
hbase.zookeeper.property.dataDir：zookeeper的位置。
/usr/local/>hbase version #有結(jié)果說(shuō)明配置正確
/usr/local/>hbase-daemon.sh start master
/usr/local/>start-hbase.sh #用于啟動(dòng)HBase笑窜，關(guān)閉HBase用stop-hbase.sh 速度相當(dāng)慢致燥，./bin/hdfs namenode -format，但是數(shù)據(jù)全部丟失
/usr/local/>hbase shell#用于打開(kāi)shell命令行模式
HBase WEB 頁(yè)面http://localhost:16010
hadoop+hbase+zookeeper安裝指南
第六步：安裝zookeeperz文件
/home/jerry/>tar -zxvf zookeeper-3.4.11.tar.gz
/home/jerry/>mv zookeeper-3.4.11 /usr/local/zookeeper
/home/jerry/>cd /usr/local/zookeeper/conf //切換到該目錄
/usr/local/zookeeper/conf >cp zoo_sample.cfg zoo.cfg//復(fù)制一份取名為zoo.cfg
/usr/local/zookeeper/conf >gedit zoo.cfg
...
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/tmp/zookeeper
dataLogDir=/tmp/zookeeper/logs
# the port at which the clients will connect
clientPort=2181
...
確保/tmp/zookeeper排截，/tmp/zookeeper/logs存在
/usr/local/>source ~/.bashrc
/usr/local/>chown -R root ./habase
/usr/local/>hbase version
/usr/local/zookeeper/conf >gedit ~/.bashrc
export ZOOKEEPER_HOME=/usr/local/zookeeper
export PATH=$PATH:$ ZOOKEEPER_HOME/bin
/usr/local/zookeeper/conf >source ~/.bashrc
cd ../bin/
/usr/local/zookeeper/bin>zkServer.sh start
在實(shí)際工作中嫌蚤，建議啟動(dòng)服務(wù)以后確定啟動(dòng)沒(méi)有問(wèn)題再關(guān)閉购撼，最后統(tǒng)一啟動(dòng)撩幽，啟動(dòng)順序?yàn)椋?/p>
haddoop
hbase
zookeep
啟動(dòng)完畢
jps
114865 NameNode
112467 JobHistoryServer
116114 HRegionServer
111637 ResourceManager
115271 SecondaryNameNode
43386 Jps
115994 HMaster
115932 HQuorumPeer
115023 DataNode
111774 NodeManager
幾種錯(cuò)誤現(xiàn)象。
1，NoNode for /hbase/master
hbase(main):005:0> status
ERROR: KeeperErrorCode = NoNode for /hbase/master
For usage try 'help "status"'
Took 8.1764 seconds
原因
HMaster進(jìn)程沒(méi)有打開(kāi)
2付枫，啟動(dòng)hadoopmei沒(méi)有datanode
cat /usr/local/hadoop/tmp/dfs/name/current/VERSION
記住clusterID
edit /usr/local/hadoop/tmp/dfs/data/current/VERSION
修改clusterID為剛才記錄的name里面的clusterID
重新啟動(dòng)hadoop
3简软，啟動(dòng)hbase后不久HMaster就關(guān)閉
如果使用后hbase自帶的zookeeper煤杀，建議單獨(dú)安裝zookeeper汇陆，否則到zookeeper主目錄下刪除hbase下所有內(nèi)容
