1.測試Oracle連接
bin/sqoop list-databases --connect jdbc:oracle:thin:@192.168.16.223:1521/orcl --username sitts --password password
2.導(dǎo)入hdfs
3.1 導(dǎo)入
bin/sqoop import --connect jdbc:oracle:thin:@192.168.16.223:1521/orcl --username sitts --password password --table SITTS.ESB_SERVICE_PARAM --target-dir /sqoop/oracle/esb_service_param --split-by PARAM_ID
3.2驗(yàn)證
hdfs dfs -ls /sqoop/oracle/esb_service_param
hdfs dfs -cat /sqoop/oracle/esb_service_param/part-m-00000
4.導(dǎo)入hbase
4.0直接新建并導(dǎo)入
bin/sqoop import --connect jdbc:oracle:thin:@192.168.16.223:1521/orcl --username sitts --password password --table SITTS.ESB_SERVICE_PARAM --split-by PARAM_ID --hbase-create-table --hbase-table orcl_esb_service_param --hbase-row-key PARAM_ID --column-family info
報(bào)錯(cuò)
16/09/07 14:05:30 INFO mapreduce.HBaseImportJob: Creating missing HBase table orcl_esb_service_param
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.hbase.HTableDescriptor.addFamily(Lorg/apache/hadoop/hbase/HColumnDescriptor;)V
原因
版本不匹配，環(huán)境上版本固定楞抡，所以選用手動新建hbase表伟众，再導(dǎo)入數(shù)據(jù)
4.1新建hbase表
hbase shell
create 'orcl_esb_service_param', 'region'
4.2導(dǎo)入mysql數(shù)據(jù)到hbase
bin/sqoop import --connect jdbc:oracle:thin:@192.168.16.223:1521/orcl --username sitts --password password --table SITTS.ESB_SERVICE_PARAM --split-by PARAM_ID --hbase-table orcl_esb_service_param --hbase-row-key PARAM_ID --column-family region
額外例子，導(dǎo)入hbase生成復(fù)合的rowKey
bin/sqoop import -D sqoop.hbase.add.row.key=true --connect jdbc:oracle:thin:@192.168.16.223:1521/orcl --username sitts --password password --table SITTS.ESB_SERVICE_PARAM --split-by PARAM_ID --hbase-table lsy_test_multiple_table --hbase-row-key PARAM_ID,SERVICE_ID --column-family data
4.3驗(yàn)證
scan 'orcl_esb_service_param'
count 'orcl_esb_service_param'
5.導(dǎo)入hive
5.1新建hive表召廷，并導(dǎo)入數(shù)據(jù)
bin/sqoop import --connect jdbc:oracle:thin:@192.168.16.223:1521/orcl --username sitts --password password --table SITTS.ESB_SERVICE_PARAM --split-by PARAM_ID --hive-import --create-hive-table --target-dir /user/hive/warehouse/orcl_esb_service_param --hive-table orcl_esb_service_param
5.2 驗(yàn)證
show tables;
select * from orcl_esb_service_param;
select count(*) from orcl_esb_service_param;
6.增量
增量到hdfs
bin/sqoop import --connect jdbc:oracle:thin:@192.168.16.223:1521/orcl --username sitts --password password --table SITTS.ESB_SERVICE_PARAM --target-dir /sqoop/oracle/esb_service_param --split-by PARAM_ID --incremental append --check-column PARAM_ID --last-value 33
返回
16/09/07 15:27:17 INFO tool.ImportTool: --incremental append
16/09/07 15:27:17 INFO tool.ImportTool: --check-column PARAM_ID
16/09/07 15:27:17 INFO tool.ImportTool: --last-value 34
16/09/07 15:27:17 INFO tool.ImportTool: (Consider saving this with 'sqoop job --create')
校驗(yàn)
hdfs dfs -cat /sqoop/oracle/esb_service_param/part-m-00004
增量到hbase
bin/sqoop import --connect jdbc:oracle:thin:@192.168.16.223:1521/orcl --username sitts --password password --table SITTS.ESB_SERVICE_PARAM --split-by PARAM_ID --hbase-table orcl_esb_service_param --hbase-row-key PARAM_ID --column-family region --incremental append --check-column PARAM_ID --last-value 33
返回
16/09/07 15:59:49 INFO tool.ImportTool: --incremental append
16/09/07 15:59:49 INFO tool.ImportTool: --check-column PARAM_ID
16/09/07 15:59:49 INFO tool.ImportTool: --last-value 34
16/09/07 15:59:49 INFO tool.ImportTool: (Consider saving this with 'sqoop job --create')
校驗(yàn)
scan 'orcl_esb_service_param'
