環(huán)境：
hbase 1.1.9集群
zookeeper-3.4.9 集群
hadoop 2.7.1集群
注：sqoop2不支持直接將mysql數(shù)據(jù)導(dǎo)入hbase,所以選擇sqoop1
一蜈漓、sqoop1安裝
1 解壓安裝
sudo tar xvfz sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz
sudo mv sqoop-1.4.6.bin__hadoop-2.0.4-alpha home/zhonglan/sqoop1/
sudo chmod -R 775 /home/zhonglan/sqoop1/
sudo chown -R zhonglan /home/zhonglan/sqoop1/
2 修改環(huán)境變量
sudo vim /etc/profile
#sqoop
export SQOOP_HOME=/ home/zhonglan/sqoop1/
export PATH=$SQOOP_HOME/bin:$PATH
source /etc/profile
3 配置sqoop的環(huán)境變量
下載mysql驅(qū)動(dòng)包穆桂，mysql-connector-java-5.1.27.jar
把jar包丟到到$SQOOP_HOME/lib下面
cp sqoop-env-template.sh sqoop-env.sh
# 指定各環(huán)境變量的實(shí)際配置
#Set path to where bin/hadoop is available
export HADOOP_COMMON_HOME=/home/zhonglan/hadoop
#Set path to where hadoop-*-core.jar is available
export HADOOP_MAPRED_HOME=/home/zhonglan/hadoop
#set the path to where bin/hbase is available
export HBASE_HOME=/home/zhonglan/hbase
#Set the path to where bin/hive is available
#export HIVE_HOME=
#Set the path for where zookeper config dir is
export ZOOCFGDIR=/home/zhonglan/zookeeper/zookeeperdata
4 測(cè)試
使用 sqoop version 查看
不用管warning部分，可在bin/configure-sqoop中注釋掉HCAT_HOME 融虽、ACCUMULO_HOME
二享完、數(shù)據(jù)導(dǎo)入
1.啟動(dòng)環(huán)境
1)啟動(dòng)hadoop集群
start-dfs.sh
start-yarn.sh
2)分別啟動(dòng)zk(注意每個(gè)節(jié)點(diǎn)的zk都要啟動(dòng))
zkServer.sh start
3)啟動(dòng)hbase镶柱，在主節(jié)點(diǎn)上運(yùn)行：
start-hbase.sh
jps
2.準(zhǔn)備數(shù)據(jù)庫
以mysql為例子
IP:222.31.81.214
用戶名:root
密碼:****
數(shù)據(jù)庫:big_data_service
表:song_topn
3.sqoop導(dǎo)入
sqoop help
Available commands:
codegen Generate code to interact with database records
create-hive-table Import a table definition into Hive
eval Evaluate a SQL statement and display the results
export Export an HDFS directory to a database table
help List available commands
import Import a table from a database to HDFS
import-all-tables Import tables from a database to HDFS
import-mainframe Import datasets from a mainframe server to HDFS
job Work with saved jobs
list-databases List available databases on a server
list-tables List available tables in a database
merge Merge results of incremental imports
metastore Run a standalone Sqoop metastore
version Display version information
查看mysql上的表
sqoop list-tables --connect jdbc:mysql://222.31.81.214/big_data_service --username root -P
在hbase上創(chuàng)建表以及列簇
create 'song_test','song'
導(dǎo)入對(duì)于的mysql數(shù)據(jù)荣瑟，id為row_key
sqoop import --connect jdbc:mysql://222.31.81.214/big_data_service --table song_topn --hbase-table song_test --column-family song --hbase-row-key id --username root -P
啟動(dòng)mapreduce任務(wù)
hbase上查看結(jié)果
scan 'song_test'
參考網(wǎng)址：
https://chu888chu888.gitbooks.io/hadoopstudy/content/Content/9/chapter0902.html
