Hive的常用函數(shù)

1谦屑、系統(tǒng)內(nèi)置函數(shù)
1.查看系統(tǒng)自帶的函數(shù)
hive> show functions;
2.顯示自帶的函數(shù)的用法
hive> desc function upper;
3.詳細(xì)顯示自帶的函數(shù)的用法
hive> desc function extended upper;

2驳糯、取值計(jì)算
1、取整函數(shù):round
語法:round(double a)
返回值:BIGINT
說明:返回double類型的整數(shù)值部分(遵循四舍五入)
select round(3.14159260 from tableName;
select round(3.5) from tableName;
create table tableName as select round(9542.158) from tableName;
2氢橙、指定精度取整函數(shù):round
語法:round(double a,int d)
返回值:DOUBLE
說明:返回指定精度d的double類型
select round(3.1415926,4) from tableName;
3酝枢、向下取整函數(shù)floor
語法:floor(double a)
返回值:BIGINT
說明:返回等于或者小于該double變量的最大的整數(shù)
select floor(3.1415926) from tableName;
select floor(25) from tableName;
4、向上取整函數(shù):ceil
語法:ceil(double a)
返回值:BIGINT
說明:返回等于或者大于該double變量的最小的整數(shù)
select ceil(3.1415926) from tableName;
select ceil(46) from tableName;
5悍手、向上取整函數(shù):ceiling
語法:ceiling(double a)
返回值:BIGINT
說明:與ceil功能相同
select ceiling(3.1415926) from tableName;
select ceiling(46) from tableName;
6帘睦、取隨機(jī)數(shù)函數(shù):rand
語法:rand(),rand(int seed)
返回值:double
說明:返回一個(gè)0到1范圍內(nèi)的隨機(jī)數(shù)。如果指定種子seed坦康,則會(huì)等到一個(gè)穩(wěn)定的隨機(jī)數(shù)序列
select rand) from tableName;
select rand() from tableName;
select rand(100) from tableName;
select rand(100) from tableName;
2.3日期函數(shù)
1竣付、UNIX時(shí)間戳轉(zhuǎn)日期函數(shù):from_unixtime
語法:from_unixtime(bigint unixtime[,string format])
返回值:string
說明:轉(zhuǎn)化UNIX時(shí)間戳(從1970-01-01 )到當(dāng)前時(shí)區(qū)的時(shí)間格式
select from_unixtime(1323308943,'yyyyMMdd') from tableName;
2、獲取當(dāng)前UNIX時(shí)間戳函數(shù)
語法:unix_timestamp()
返回值:bigint
說明:獲得當(dāng)時(shí)時(shí)區(qū)的UNIX時(shí)間戳
select unix_timestamp() from tableName;
3滞欠、日期轉(zhuǎn)UNIX時(shí)間戳函數(shù):unix_timestamp
語法:unix_timestam(string date)
返回值:bigint
說明:轉(zhuǎn)換格式為'yyyy-MM-dd HH:mm:ss"的日期到UMIX時(shí)間戳古胆。如果轉(zhuǎn)換失敗,則返回0筛璧、
select unix_timestamp('2011-12-07 13:01:03') from tableName;
4逸绎、指定格式日期轉(zhuǎn)UNIX時(shí)間戳函數(shù):unix_timestamp
語法:unix_timestamp(string date,string pattern)
返回值:bigint
說明:轉(zhuǎn)換pattern格式的日期到UNiX時(shí)間戳惹恃。如果轉(zhuǎn)化失敗,則返回0.
select unix_timestamp('20111207 13:01:03','yyyyMMdd HH:mm:ss') from tableName;
5棺牧、日期時(shí)間轉(zhuǎn)日期函數(shù):to_date
語法:to_date(string datetime)
返回值:string
說明:返回日期時(shí)間字段中的日期部分巫糙。
select to_date("2011-12-08 10:03:01') from tableName
6、日期轉(zhuǎn)年函數(shù):year
語法:year(string date)
返回值:int
說明:返回日期中的年
select year('2011-12-08 10:03:01') from tableName;

select year('2012-12-08') from tableName;
7陨帆、日期轉(zhuǎn)月函數(shù):month
語法:month(string date)
返回值:int
說明:返回date或datetime中的月份
select month('2011-12-08 10:03:01') from tableName;
select month('2011-08-08') from tableName;

8曲秉、日期轉(zhuǎn)天函數(shù):day
語法:day(strign date)
返回值:int
說明:返回值日期中的天
select day('2011-12-08 10:03:01') from tableName;
select day('2011-12-24') from tableName;

9、日期轉(zhuǎn)小時(shí)函數(shù): hour
  • 語法: hour (string date)
  • 返回值: int
  • 說明: 返回日期中的小時(shí)疲牵。
hive> select hour('2011-12-08 10:03:01') from tableName;
10
10、日期轉(zhuǎn)分鐘函數(shù): minute
  • 語法: minute (string date)
  • 返回值: int
  • 說明: 返回日期中的分鐘榆鼠。
hive> select minute('2011-12-08 10:03:01') from tableName;
3

-- second 返回秒
hive> select second('2011-12-08 10:03:01') from tableName;
1
12纲爸、日期轉(zhuǎn)周函數(shù): weekofyear
  • 語法: weekofyear (string date)
  • 返回值: int
  • 說明: 返回日期在當(dāng)前的周數(shù)。
hive> select weekofyear('2011-12-08 10:03:01') from tableName;
49
13妆够、日期比較函數(shù): datediff
  • 語法: datediff(string enddate, string startdate)
  • 返回值: int
  • 說明: 返回結(jié)束日期減去開始日期的天數(shù)识啦。
hive> select datediff('2012-12-08','2012-05-09') from tableName;
213
14、日期增加函數(shù): date_add
  • 語法: date_add(string startdate, int days)
  • 返回值: string
  • 說明: 返回開始日期startdate增加days天后的日期神妹。
hive> select date_add('2012-12-08',10) from tableName;
2012-12-18
15颓哮、日期減少函數(shù): date_sub
  • 語法: date_sub (string startdate, int days)
  • 返回值: string
  • 說明: 返回開始日期startdate減少days天后的日期。
hive> select date_sub('2012-12-08',10) from tableName;
2012-11-28

2.4 條件函數(shù)(重點(diǎn))

1鸵荠、if函數(shù):if

1、If函數(shù): if
  • 語法: if(boolean testCondition, T valueTrue, T valueFalseOrNull)
  • 返回值: T
  • 說明: 當(dāng)條件testCondition為TRUE時(shí),返回valueTrue鲫咽;否則返回valueFalseOrNull
    select if(1=2,100,200) from tableName;
    select if(1=1,100,200) from tableName;
    2劫恒、非空查找函數(shù):COALESCE
    語法:COALESCE(T v1,T v2,..)
    返回值:T
    說明:返回參數(shù)中的第一個(gè)非空值;如果所有值都為NULL庸疾,那么返回NULL
    select COALESCE(null,'100','50') from tableName;
    3乍楚、條件判斷函數(shù):CASE
    語法:CASE a WHEN b THEN c [WHEN d THEN e] * [ELSE f] END
    返回值:T
    說明:如果a等于b,那么返回c;如果a等于d届慈,那么返回e徒溪;否則返回f
    select case 100 when 50 then 'tom' when 100 then 'mary' else 'tim' end from tableName;

select case 200 when 50 then 'tom' when 100 then 'mary' else 'tim' end from tableName;

4、條件判斷函數(shù):CASE
語法:CASE WHEN a THEN b [WHEN c THEN d] * [ELSSE e] END
返回值:T
說明:如果a為TRUE金顿,則返回b臊泌;如果c為TRUE,則返回d;否則返回e
select case when 1=2 then 'tom' when 2=2 then 'mary' else 'tim' end from tableName;
select case when 1=1 then 'tom' when 2=2 then 'mary' else 'tim' end from tableName;
2.5字符串函數(shù)
1串绩、字符串長度函數(shù)length
語法:length(string A)
返回值:int
說明:返回字符串A的長度
select length('abcedfg') from tableName;
2缺虐、字符串反轉(zhuǎn)函數(shù):reverse
語法:reverse(string A)
返回值:string
說明:返回字符串A的反轉(zhuǎn)結(jié)果
select reverse('abcedfg') from tableName;
3、字符串連接函數(shù):concat
語法:concat(string A,string B..)
返回值:string
說明:返回輸入字符串連接后的結(jié)果礁凡,支持任意個(gè)輸入字符串
select concat('abc','def','gh') from tableName;
4高氮、字符串連接并指定字符串分隔符:concat_ws
語法:concat_ws(string SEP,string A,string B..)
返回值:string
說明:返回輸入字符串連接后的結(jié)果慧妄,SEP表示各個(gè)字符串間的分隔符
select concat_ws(',','abc','def','gh') from tableName;
5、字符串截取函數(shù):substr
語法:substr(string A,int start),substring(string A,int start)
返回值:string
說明:返回字符串A從start位置到結(jié)尾的字符串
select substr('abcde',3) from tableName;
select substring('abcde',3) from tableName;
select substr('abcde',-1) from tableName;
6剪芍、字符串截取函數(shù):substr,substring
語法:substr(string A,int start,int len),substring(string a,int start,int len)
返回值:string
說明:返回字符串A從start位置開始塞淹,長度為len的字符串
select substr('abcde',3,2) from tableName;
select substring('abcde',3,2) from tableName;
select substring('abcde',-3,2) from tableName;

7、字符串轉(zhuǎn)大寫函數(shù):upper,ucase
語法:upper(string A) ucase(string A)
返回值:string
說明:返回字符串A的大寫格式
select upper('abSed') from tableName;
select ucase('abSED') from tableName;

8罪裹、字符串轉(zhuǎn)小寫函數(shù):lower,lcase
語法:lower(string A) lcase(string A)
返回值:string
說明:返回字符串A的小寫格式
select lower('abSED') from tableName;
select ;case('abSED') from tableName;

9饱普、去空格函數(shù):trim
語法:trim(string A)
返回值:string
說明:去除字符串兩邊的空格
select trim(' ab c ') from tableNmae;

10、url解析函數(shù) parse_url
語法:parse_url(string urlString,string partToExtract[,string keyToExtract])
返回值:string
說明:返回url中指定的部分状共。partToExtract的有效值為:HOSt套耕,PATH,QUERY峡继,REF冯袍,POTOCOL,AUTHORITY,FILE,and USERINFO
select parse_url
('https://www.tableName.com/path1/p.php?k1=v1&k2=v2#Ref1', 'HOST')
from tableName;
select parse_url
('https://www.tableName.com/path1/p.php?k1=v1&k2=v2#Ref1', 'QUERY', 'k1')
from tableName;
11碾牌、json解析get_json_object
語法:get_json_object(string json_string,string path)
返回值:string
說明:解析json的字符串json_string康愤,返回path指定的內(nèi)容。如果輸入的json字符串無效舶吗,那么返回NULL
select get_json_object('{"store":{"fruit":[{"weight":8,"type":"apple"},{"weight":9,"type":"pear"}], "bicycle":{"price":19.95,"color":"red"} },"email":"amy@only_for_json_udf_test.net","owner":"amy"}','$.owner') from tableName;

12征冷、重復(fù)字符串函數(shù):repeat
語法:repeat(string str,int n)
返回值:string
說明:返回重復(fù)n次后的str字符串
select repeat('abc',5) from tableName;

13、分割字符串函數(shù):split

  • 語法: split(string str, string pat)
  • 返回值: array
  • 說明: 按照pat字符串分割str誓琼,會(huì)返回分割后的字符串?dāng)?shù)組
hive> select split('abtcdtef','t') from tableName;
["ab","cd","ef"]

2.6 集合統(tǒng)計(jì)函數(shù)

1检激、個(gè)數(shù)統(tǒng)計(jì)函數(shù): count
  • 語法: count(*), count(expr), count(DISTINCT expr[, expr_.])
  • 返回值:Int
  • 說明: count(*)統(tǒng)計(jì)檢索出的行的個(gè)數(shù),包括NULL值的行踊赠;count(expr)返回指定字段的非空值的個(gè)數(shù)呵扛;count(DISTINCT
    expr[, expr_.])返回指定字段的不同的非空值的個(gè)數(shù)
hive> select count(*) from tableName;
20
hive> select count(distinct t) from tableName;
10
2、總和統(tǒng)計(jì)函數(shù): sum
  • 語法: sum(col), sum(DISTINCT col)
  • 返回值: double
  • 說明: sum(col)統(tǒng)計(jì)結(jié)果集中col的相加的結(jié)果筐带;sum(DISTINCT col)統(tǒng)計(jì)結(jié)果中col不同值相加的結(jié)果
hive> select sum(t) from tableName;
100
hive> select sum(distinct t) from tableName;
70
3今穿、平均值統(tǒng)計(jì)函數(shù): avg
  • 語法: avg(col), avg(DISTINCT col)
  • 返回值: double
  • 說明: avg(col)統(tǒng)計(jì)結(jié)果集中col的平均值;avg(DISTINCT col)統(tǒng)計(jì)結(jié)果中col不同值相加的平均值
hive> select avg(t) from tableName;
50
hive> select avg (distinct t) from tableName;
30
4伦籍、最小值統(tǒng)計(jì)函數(shù): min
  • 語法: min(col)
  • 返回值: double
  • 說明: 統(tǒng)計(jì)結(jié)果集中col字段的最小值
hive> select min(t) from tableName;
20
5蓝晒、最大值統(tǒng)計(jì)函數(shù): max
  • 語法: max(col)
  • 返回值: double
  • 說明: 統(tǒng)計(jì)結(jié)果集中col字段的最大值
    hive> select max(t) from tableName;
    120

2.7 復(fù)合類型構(gòu)建函數(shù)

1、Map類型構(gòu)建: map
  • 語法: map (key1, value1, key2, value2, …)
  • 說明:根據(jù)輸入的key和value對構(gòu)建map類型
    --建表
    create table score_map(name string ,score map<string,int> ) row format delimited fields terminated by '\t'
    collection items terminated by ','
    map keys terminated by ':";
    -- 創(chuàng)建數(shù)據(jù)內(nèi)容如下并加載數(shù)據(jù)
    cd /kkb/install/hivedatas/
    vim score_map.txt

zhangsan 數(shù)學(xué):80,語文:89,英語:95
lisi 語文:60,數(shù)學(xué):80,英語:99

-- 加載數(shù)據(jù)到hive表當(dāng)中去
load data local inpath '/kkb/install/hivedatas/score_map.txt' overwrite into table score_map;

-- map結(jié)構(gòu)數(shù)據(jù)訪問:
-- 獲取所有的value:
select name,map_values(score) from score_map;

-- 獲取所有的key:
select name,map_keys(score) from score_map;

-- 按照key來進(jìn)行獲取value值
select name,score["數(shù)學(xué)"] from score_map;

-- 查看map元素個(gè)數(shù)
select name,size(score) from score_map;

-- 構(gòu)建一個(gè)map
select map(1, 'zs', 2, 'lisi');

2帖鸦、Struct類型構(gòu)建:struct
語法:struct(val1,val2,val3,..)
說明:根據(jù)輸入的參數(shù)構(gòu)建結(jié)構(gòu)體struct類型芝薇,似于C語言中的結(jié)構(gòu)體,內(nèi)部數(shù)據(jù)通過X.X來獲取作儿,
--創(chuàng)建struct表
create table movie_score(name string,info struct<number:int,score:float>)
row format delimited fields terminated by "\t"
collection items terminated by ":";
--加載數(shù)據(jù)
cd /kkb/install/hivedatas/
vim struct.txt
--電影ABC洛二,有12345評價(jià)過,打分7.4分
ABC 1254:7.4
DEF 256:4.9
XYZ 456:5.4
--加載數(shù)據(jù)
load data local inpath '/kkb/install/hivedatas/struct.txt" overwrite into table move_score;
--hive當(dāng)中查詢數(shù)據(jù)
select * from move_score;
select info.number,info.score from movie_score;

select struct(1,'anzhulababy','moon',1.68);

3、Array類型構(gòu)建:array
語法:array(val1,val2,..)
說明:根據(jù)輸入的參數(shù)構(gòu)建數(shù)組array類型
create table person(name string,work_locations array<string>) row format delitmited fields terminated by '\t' collection iterms terminated by ',';
-- 加載數(shù)據(jù)到person表當(dāng)中去
cd /kkb/install/hivedatas/
vim person.txt

-- 數(shù)據(jù)內(nèi)容格式如下
biansutao beijing,shanghai,tianjin,hangzhou
linan changchun,chengdu,wuhan

-- 加載數(shù)據(jù)
hive > load data local inpath '/kkb/install/hivedatas/person.txt' overwrite into table person;

--查詢所有數(shù)據(jù)
select * from person;
--按照下表索引進(jìn)行查詢
select work_locations[0] from person;
--查詢所有集合數(shù)據(jù)
select work_locations from person;
--查詢元素格式
select size(work_locations) from person;
--構(gòu)建 array
select array(1,2,1);
select array(1,'a',1.0);
select array(1,2,1.0);

2.8 復(fù)雜類型長度統(tǒng)計(jì)函數(shù)

1晾嘶、Map類型長度函數(shù): size(Map<k .V>)
  • 語法: size(Map<k .V>)
  • 返回值: int
  • 說明: 返回map類型的長度
hive> select size(map(1, 'zs', 2, 'anzhulababy')) from tableName;
2
2妓雾、array類型長度函數(shù): size(Array<T>)
  • 語法: size(Array<T>)
  • 返回值: int
  • 說明: 返回array類型的長度
hive> select size(t) from arr_table2;
4
3、類型轉(zhuǎn)換函數(shù)
  • 類型轉(zhuǎn)換函數(shù): cast
  • 語法: cast(expr as <type>)
  • 返回值: Expected "=" to follow "type"
  • 說明: 返回轉(zhuǎn)換后的數(shù)據(jù)類型
hive> select cast('1' as bigint) from tableName;
1

2.9 行轉(zhuǎn)列

1垒迂、相關(guān)函數(shù)說明
  • CONCAT(string A/col, string B/col…):返回輸入字符串連接后的結(jié)果械姻,支持任意個(gè)輸入字符串;
  • CONCAT_WS(separator, str1, str2,...):它是一個(gè)特殊形式的 CONCAT()。
    • 第一個(gè)參數(shù)剩余參數(shù)間的分隔符机断。分隔符可以是與剩余參數(shù)一樣的字符串楷拳。如果分隔符是 NULL,返回值也將為 NULL吏奸。
    • 這個(gè)函數(shù)會(huì)跳過分隔符參數(shù)后的任何 NULL 和空字符串欢揖。分隔符將被加到被連接的字符串之間;
  • COLLECT_SET(col):函數(shù)只接受基本數(shù)據(jù)類型,它的主要作用是將某字段的值進(jìn)行去重匯總苦丁,產(chǎn)生array類型字段浸颓。
2、數(shù)據(jù)準(zhǔn)備
  • 數(shù)據(jù)準(zhǔn)備
name constellation blood_type
孫悟空 白羊座 A
老王 射手座 A
宋宋 白羊座 B
豬八戒 白羊座 A
按住啦baby 射手座 A
3旺拉、需求
  • 把星座和血型一樣的人歸類到一起。結(jié)果如下:
射手座,A            老王|按住啦baby
白羊座,A            孫悟空|豬八戒
白羊座,B            宋宋
4棵磷、創(chuàng)建表數(shù)據(jù)文件
  • node03服務(wù)器執(zhí)行以下命令創(chuàng)建文件蛾狗,注意數(shù)據(jù)使用\t進(jìn)行分割
cd /kkb/install/hivedatas
vim constellation.txt
孫悟空 白羊座 A
老王  射手座 A
宋宋  白羊座 B       
豬八戒 白羊座 A
鳳姐  射手座 A
5、創(chuàng)建hive表并導(dǎo)入數(shù)據(jù)
  • 創(chuàng)建hive表并加載數(shù)據(jù)
    create table person_info(name string,constellation string,blood_type string) row format delimited fields terminated by '\t';
    加載數(shù)據(jù)
    load data local inpath '/kkb/install/hivedatas/constellation.txt' into table person_info;
    6仪媒、按需求查詢數(shù)據(jù)
    select t1.base,concat_ws('|',collect_st(t1.name)) name from (select name,concat(constellation,",",blood_type) base from person_info) t1 group by t1.base;
    2.10列轉(zhuǎn)行
    1沉桌、函數(shù)說明
    EXPLODE(col):將hive一列中復(fù)雜的array或者map結(jié)構(gòu)拆分成多行。
    LATERAL VIEW
    用法:LATERAL VIEW udft(expression) tableAlias AS columnAlias
    解釋:用于和split,explode等UDTF一起使用算吩,它能夠?qū)⒁涣袛?shù)據(jù)拆成多行數(shù)據(jù)留凭,在此基礎(chǔ)上可以對拆分后的數(shù)據(jù)進(jìn)行聚合。
    2偎巢、數(shù)據(jù)準(zhǔn)備
    數(shù)據(jù)內(nèi)容如下蔼夜,字段之間都是使用\t進(jìn)行分割
    cd /kkb/install/hivedatas
    vim move.txt

《疑犯追蹤》 懸疑,動(dòng)作,科幻,劇情
《Lie to me》 懸疑,警匪,動(dòng)作,心理,劇情
《戰(zhàn)狼2》 戰(zhàn)爭,動(dòng)作,災(zāi)難
3、需求
將電影分類中的數(shù)據(jù)展開压昼。結(jié)果如下:
《疑犯追蹤》 懸疑
《疑犯追蹤》 動(dòng)作
《疑犯追蹤》 科幻
《疑犯追蹤》 劇情
《Lie to me》 懸疑
《Lie to me》 警匪
《Lie to me》 動(dòng)作
《Lie to me》 心理
《Lie to me》 劇情
《戰(zhàn)狼2》 戰(zhàn)爭
《戰(zhàn)狼2》 動(dòng)作
《戰(zhàn)狼2》 災(zāi)難
4求冷、創(chuàng)建hive表并導(dǎo)入數(shù)據(jù)
創(chuàng)建hive表
create table movie_info(movie string,categroy array<string>) row format delimited fields terminated by "\t" collection items terminated by ",";
加載數(shù)據(jù)
load data local inpath "/kkb/install/hivedatas/movie.txt" into table move_info;
5、按需求查詢數(shù)據(jù)
select movie,catagory_name from move_info
lateral view exploade(cate gory) table_tmp as category_name;

2.11 lateral view窍霞、explode練習(xí)

1匠题、explode函數(shù)將Map和Array字段數(shù)據(jù)進(jìn)行拆分
  • explode還可以用于將hive一列中復(fù)雜的array或者map結(jié)構(gòu)拆分成多行
-- 需求:現(xiàn)在有數(shù)據(jù)格式如下
zhangsan    child1,child2,child3,child4 k1:v1,k2:v2
lisi    child5,child6,child7,child8  k3:v3,k4:v4

-- 字段之間使用\t分割,需求將所有的child進(jìn)行拆開成為一列
+----------+--+
| mychild  |
+----------+--+
| child1   |
| child2   |
| child3   |
| child4   |
| child5   |
| child6   |
| child7   |
| child8   |
+----------+--+

-- 將map的key和value也進(jìn)行拆開但金,成為如下結(jié)果
+-----------+-------------+--+
| mymapkey  | mymapvalue  |
+-----------+-------------+--+
| k1        | v1          |
| k2        | v2          |
| k3        | v3          |
| k4        | v4          |
+-----------+-------------+--+

第一步:創(chuàng)建hive數(shù)據(jù)庫
創(chuàng)建hive數(shù)據(jù)庫
create database hive_explode;
use hive_explode;
第二步:創(chuàng)建hive表
create table hive_explode.t3(name string,children array<string>,address Map<string,string>) row format delimited fields terminated by '\t' collection items terminated by ',' map keys terminated by ':' stored as textFile;
第三步:加載數(shù)據(jù)
node03執(zhí)行以下命令創(chuàng)建表數(shù)據(jù)文件
cd /kkb/install/hivedatas/
vim maparray
-- 數(shù)據(jù)內(nèi)容格式如下
zhangsan child1,child2,child3,child4 k1:v1,k2:v2
lisi child5,child6,child7,child8 k3:v3,k4:v4
hive表當(dāng)中加載數(shù)據(jù)
hive (hive_explode)> load data local inpath '/kkb/install/hivedatas/maparray' into table hive_explode.t3;

第四步:使用explode將hive當(dāng)中數(shù)據(jù)拆開
  • 將array當(dāng)中的數(shù)據(jù)拆分開
    select explode(children) AS myChild FROm hive_explode.t3;
    將map當(dāng)中的數(shù)據(jù)拆分開
    select explode(address) AS (myMapKey,myMapValue) FROM hive_explode.t3;

2韭山、使用explode拆分json字符串
需求:現(xiàn)在有一些數(shù)據(jù)格式如下:
a:shandong,b:beijing,c:hebei|1,2,3,4,5,6,7,8,9|[{"source":"7fresh","monthSales":4900,"userCount":1900,"score":"9.9"},{"source":"jd","monthSales":2090,"userCount":78981,"score":"9.8"},{"source":"jdmart","monthSales":6987,"userCount":1600,"score":"9.0"}]
其中字段與字段之間的分隔符是|
我們要解析得到所有的monthSales對應(yīng)的值為以下這一列(行轉(zhuǎn)列)
第一步:創(chuàng)建hive表
create table hive_explode.explode_lateral_view (area string,goods_id string,sale_info string) ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' STORED AS textfile;
第二步:準(zhǔn)備數(shù)據(jù)并加載數(shù)據(jù)
準(zhǔn)備數(shù)據(jù)如下
cd /kkb/install/hivedatas
vim explode_json
a:shandong,b:beijing,c:hebei|1,2,3,4,5,6,7,8,9|[{"source":"7fresh","monthSales":4900,"userCount":1900,"score":"9.9"},{"source":"jd","monthSales":2090,"userCount":78981,"score":"9.8"},{"source":"jdmart","monthSales":6987,"userCount":1600,"score":"9.0"}]

加載數(shù)據(jù)到hive表當(dāng)中去
load data local inpath '/kkb/install/hivedatas/explode_json' overwrite into table hive_explode.explode_lateral_view;

第三步:使用explode拆分Array
select explode(split(goods_id,',')) as goods_id from hive_explode.explode_lateral_view;

第四步:使用explode拆解Map
select explode(split(area,',')) as area from hive_explode.explode_lateral_view;
第五步:拆解json字段
select explode(split(regexp_replace(regexp_replace(sale_info,'\[\{',''),'}]',''),'},\{')) as sale_info from hive_explode.explode_lateral_view;

然后我們想用get_json_object來獲取key為monthSales的數(shù)據(jù):
select get_json_object(explode(split(regexp_replace(regexp_replace(sale_info,'\[\{',''),'}]',''),'},\{')),'$.monthSales') as sale_info from hive_explode.explode_lateral_view;

-- 然后出現(xiàn)異常FAILED: SemanticException [Error 10081]: UDTF's are not supported outside the SELECT clause, nor nested in expressions
-- UDTF explode不能寫在別的函數(shù)內(nèi)

-- 如果你這么寫,想查兩個(gè)字段,select explode(split(area,',')) as area,good_id from explode_lateral_view;
-- 會(huì)報(bào)錯(cuò)FAILED: SemanticException 1:40 Only a single expression in the SELECT clause is supported with UDTF's. Error encountered near token 'good_id'
-- 使用UDTF的時(shí)候钱磅,只支持一個(gè)字段梦裂,這時(shí)候就需要LATERAL VIEW出場了
3、配合LATERAL VIEW使用
lateral view用于和split续搀、explode等UDTF一起使用塞琼,能將一行數(shù)據(jù)拆分成多行數(shù)據(jù)
在此基礎(chǔ)上可以對拆分的數(shù)據(jù)進(jìn)行聚合
later view首先為原始表的每行調(diào)用UDTF,UDTF會(huì)把一行拆分成一行或者多行,lateral view在把結(jié)果組合禁舷,產(chǎn)生一個(gè)支持別名表的虛擬表彪杉。
配合lateral view查詢多個(gè)字段
select goods_id2,sale_info from explode_lateral_view LATERAL VIEW explode(split(goods_id,',')) goods as goods_id2;
其中LATeRAL VIEW explode(split(goods_id,','))goods相當(dāng)于一個(gè)虛擬表,與原表explode_laterral_view笛卡爾積關(guān)聯(lián)牵咙。
也可以多重使用派近,如下,也是三個(gè)笛卡爾積的結(jié)果
select goods_id2,sale_info,area2 from explode_lateral_view LATERAL VIEW explode(split(goods_id,',')) goods as goods_id2
LATERAL VIEW explode(split(area,','))area as area2;
最終洁桌,我們可以通過下面的句子渴丸,把這個(gè)json格式的一行數(shù)據(jù),完全轉(zhuǎn)換成二維表的方式展現(xiàn)
select get_json_object(concat('{',sale_info_1,'}'),'.source') as source, get_json_object(concat('{',sale_info_1,'}'),'.monthSales') as monthSales, get_json_object(concat('{',sale_info_1,'}'),'.userCount') as userCount, get_json_object(concat('{',sale_info_1,'}'),'.score') as score
from explode_lateral_view LATERAL VIEW explode(split(regexp_replace(regexp_replace(sale_info,'\[\{',''),'}]',''),'},\{'))sale_info as sale_info_1;

總結(jié):Lateral View通常和UDTF一起出現(xiàn)另凌,為了解決UDTF不允許在select字段的問題谱轨。
Multiple Lateral View可以實(shí)現(xiàn)類似笛卡爾乘積。
Outer關(guān)鍵字可以把不輸出的UDTF的空結(jié)果吠谢,輸出成NULL土童,防止丟失數(shù)據(jù)。

2.12 reflect函數(shù)
reflect函數(shù)可以支持在sql中調(diào)用java中的自帶函數(shù)工坊,秒殺一切udf函數(shù)献汗。
1、使用java.lang.Math當(dāng)中的Max求兩列中最大值
創(chuàng)建hive表
create table test_udf(col1 int,col2 int) row format delimited fields terminated by ',';
準(zhǔn)備數(shù)據(jù)并加載數(shù)據(jù)
cd /kkb/install/hivedatas
vim test_udf

1,2
4,3
6,4
7,5
5,6
加載數(shù)據(jù)
load data local inpath '/kkb/install/hivedatas/test_udf' overwrite into table test_udf;
使用java.lang.Math當(dāng)中的Max求兩列當(dāng)中的最大值
select reflect("java.lang.Math","max",col1,col2) from test_udf;
2王污、不同記錄執(zhí)行不同的java內(nèi)置函數(shù)
創(chuàng)建hive表
create table test_udf2(class_name string,method_name string,col1 int,col2 int) row format delimited fields terminated by ',';
準(zhǔn)備數(shù)據(jù)
cd /export/servers/hivedatas
vim test_udf2

java.lang.Math,min,1,2
java.lang.Math,max,2,3
加載數(shù)據(jù)
load data local inpath '/kkb/install/hivedatas/test_udf2' overwrite into table test_udf2;
執(zhí)行查詢
select reflect(class_name,method_name,col1,col2) from test_udf2;
3罢吃、判斷是否為數(shù)字

  • 使用apache commons中的函數(shù),commons下的jar已經(jīng)包含在hadoop的classpath中昭齐,所以可以直接使用尿招。
  • 使用方式如下:
    hive (hive_explode)> select reflect("org.apache.commons.lang.math.NumberUtils", "isNumber", "123");

2.13 分析函數(shù)—分組求topN

1、分析函數(shù)的作用
  • 對于一些比較復(fù)雜的數(shù)據(jù)求取過程司浪,我們可能就要用到分析函數(shù)

  • 分析函數(shù)主要用于==分組求topN或者求取百分比泊业,或者進(jìn)行數(shù)據(jù)的切片==等等,我們都可以使用分析函數(shù)來解決

2啊易、常用的分析函數(shù)

1吁伺、ROW_NUMBER():

  • 從1開始,按照順序租谈,給分組內(nèi)的記錄加序列篮奄;

    • 比如捆愁,按照pv降序排列,生成分組內(nèi)每天的pv名次,ROW_NUMBER()的應(yīng)用場景非常多

    • 再比如窟却,獲取分組內(nèi)排序第一的記錄;

    • 獲取一個(gè)session中的第一條refer等昼丑。

2、RANK() :

  • 生成數(shù)據(jù)項(xiàng)在分組中的排名夸赫,排名相等會(huì)在名次中留下空位

3菩帝、DENSE_RANK() :

  • 生成數(shù)據(jù)項(xiàng)在分組中的排名,排名相等會(huì)在名次中不會(huì)留下空位

4茬腿、CUME_DIST :

  • 小于等于當(dāng)前值的行數(shù)/分組內(nèi)總行數(shù)呼奢。比如,統(tǒng)計(jì)小于等于當(dāng)前薪水的人數(shù)切平,所占總?cè)藬?shù)的比例

5握础、PERCENT_RANK :

  • 分組內(nèi)當(dāng)前行的RANK值/分組內(nèi)總行數(shù)

6、NTILE(n) :

  • 用于將分組數(shù)據(jù)按照順序切分成n片悴品,返回當(dāng)前切片值

  • 如果切片不均勻禀综,默認(rèn)增加第一個(gè)切片的分布。

  • NTILE不支持ROWS BETWEEN苔严,比如 NTILE(2) OVER(PARTITION BY cookieid ORDER BY createtime ROWS BETWEEN 3 PRECEDING AND CURRENT ROW)

3定枷、需求描述
  • 現(xiàn)有數(shù)據(jù)內(nèi)容格式如下,分別對應(yīng)三個(gè)字段届氢,cookieid依鸥,createtime ,pv

  • 求取每個(gè)cookie訪問pv前三名的數(shù)據(jù)記錄悼沈,其實(shí)就是分組求topN,求取每組當(dāng)中的前三個(gè)值

<pre class="md-fences md-end-block ty-contain-cm modeLoaded" lang="" style="box-sizing: border-box; overflow: visible; font-family: var(--monospace); font-size: 0.9em; display: block; break-inside: avoid; text-align: left; white-space: normal; background-image: inherit; background-position: inherit; background-size: inherit; background-repeat: inherit; background-attachment: inherit; background-origin: inherit; background-clip: inherit; background-color: rgb(248, 248, 248); position: relative !important; border: 1px solid rgb(231, 234, 237); border-radius: 3px; padding: 8px 4px 6px; margin-bottom: 15px; margin-top: 15px; width: inherit; color: rgb(51, 51, 51); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;" spellcheck="false" mdtype="fences" cid="n888">cookie1,2015-04-10,1
cookie1,2015-04-11,5
cookie1,2015-04-12,7
cookie1,2015-04-13,3
cookie1,2015-04-14,2
cookie1,2015-04-15,4
cookie1,2015-04-16,4
cookie2,2015-04-10,2
cookie2,2015-04-11,3
cookie2,2015-04-12,5
cookie2,2015-04-13,6
cookie2,2015-04-14,3
cookie2,2015-04-15,9
cookie2,2015-04-16,7</pre>

第一步:創(chuàng)建數(shù)據(jù)庫表
  • 在hive當(dāng)中創(chuàng)建數(shù)據(jù)庫表

<pre class="md-fences md-end-block ty-contain-cm modeLoaded" lang="sql" style="box-sizing: border-box; overflow: visible; font-family: var(--monospace); font-size: 0.9em; display: block; break-inside: avoid; text-align: left; white-space: normal; background-image: inherit; background-position: inherit; background-size: inherit; background-repeat: inherit; background-attachment: inherit; background-origin: inherit; background-clip: inherit; background-color: rgb(248, 248, 248); position: relative !important; border: 1px solid rgb(231, 234, 237); border-radius: 3px; padding: 8px 4px 6px; margin-bottom: 15px; margin-top: 15px; width: inherit; color: rgb(51, 51, 51); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;" spellcheck="false" mdtype="fences" cid="n893">CREATE EXTERNAL TABLE cookie_pv (
cookieid string,
createtime string,
pv INT
) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' ;</pre>

第二步:準(zhǔn)備數(shù)據(jù)并加載
  • node03執(zhí)行以下命令姐扮,創(chuàng)建數(shù)據(jù)絮供,并加載到hive表當(dāng)中去

<pre class="md-fences md-end-block ty-contain-cm modeLoaded" lang="shell" style="box-sizing: border-box; overflow: visible; font-family: var(--monospace); font-size: 0.9em; display: block; break-inside: avoid; text-align: left; white-space: normal; background-image: inherit; background-position: inherit; background-size: inherit; background-repeat: inherit; background-attachment: inherit; background-origin: inherit; background-clip: inherit; background-color: rgb(248, 248, 248); position: relative !important; border: 1px solid rgb(231, 234, 237); border-radius: 3px; padding: 8px 4px 6px; margin-bottom: 15px; margin-top: 15px; width: inherit; color: rgb(51, 51, 51); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;" spellcheck="false" mdtype="fences" cid="n898">cd /kkb/install/hivedatas
vim cookiepv.txt
?
cookie1,2015-04-10,1
cookie1,2015-04-11,5
cookie1,2015-04-12,7
cookie1,2015-04-13,3
cookie1,2015-04-14,2
cookie1,2015-04-15,4
cookie1,2015-04-16,4
cookie2,2015-04-10,2
cookie2,2015-04-11,3
cookie2,2015-04-12,5
cookie2,2015-04-13,6
cookie2,2015-04-14,3
cookie2,2015-04-15,9
cookie2,2015-04-16,7</pre>

  • 加載數(shù)據(jù)到hive表當(dāng)中去

<pre class="md-fences md-end-block ty-contain-cm modeLoaded" lang="sql" style="box-sizing: border-box; overflow: visible; font-family: var(--monospace); font-size: 0.9em; display: block; break-inside: avoid; text-align: left; white-space: normal; background-image: inherit; background-position: inherit; background-size: inherit; background-repeat: inherit; background-attachment: inherit; background-origin: inherit; background-clip: inherit; background-color: rgb(248, 248, 248); position: relative !important; border: 1px solid rgb(231, 234, 237); border-radius: 3px; padding: 8px 4px 6px; margin-bottom: 15px; margin-top: 15px; width: inherit; color: rgb(51, 51, 51); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;" spellcheck="false" mdtype="fences" cid="n902">load data local inpath '/kkb/install/hivedatas/cookiepv.txt' overwrite into table cookie_pv;</pre>

第三步:使用分析函數(shù)來求取每個(gè)cookie訪問PV的前三條記錄

<pre class="md-fences md-end-block ty-contain-cm modeLoaded" lang="sql" style="box-sizing: border-box; overflow: visible; font-family: var(--monospace); font-size: 0.9em; display: block; break-inside: avoid; text-align: left; white-space: normal; background-image: inherit; background-position: inherit; background-size: inherit; background-repeat: inherit; background-attachment: inherit; background-origin: inherit; background-clip: inherit; background-color: rgb(248, 248, 248); position: relative !important; border: 1px solid rgb(231, 234, 237); border-radius: 3px; padding: 8px 4px 6px; margin-bottom: 15px; margin-top: 15px; width: inherit; color: rgb(51, 51, 51); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;" spellcheck="false" mdtype="fences" cid="n904">select * from (
SELECT
cookieid,
createtime,
pv,
RANK() OVER(PARTITION BY cookieid ORDER BY pv desc) AS rn1,
DENSE_RANK() OVER(PARTITION BY cookieid ORDER BY pv desc) AS rn2,
ROW_NUMBER() OVER(PARTITION BY cookieid ORDER BY pv DESC) AS rn3
FROM cookie_pv
) temp where temp.rn1 <= 3;</pre>

2.14 hive自定義函數(shù)

1、自定義函數(shù)的基本介紹
  • Hive 自帶了一些函數(shù)茶敏,比如:max/min等壤靶,但是數(shù)量有限,自己可以通過自定義UDF來方便的擴(kuò)展惊搏。

  • 當(dāng)Hive提供的內(nèi)置函數(shù)無法滿足你的業(yè)務(wù)處理需要時(shí)贮乳,此時(shí)就可以考慮使用用戶自定義函數(shù)(UDF:user-defined function)

  • 根據(jù)用戶自定義函數(shù)類別分為以下三種:

    • UDF(User-Defined-Function) 一進(jìn)一出

    • UDAF(User-Defined Aggregation Function) 聚集函數(shù),多進(jìn)一出恬惯,類似于:count/max/min

    • UDTF(User-Defined Table-Generating Functions) 一進(jìn)多出向拆,如lateral view explode()

如lateral view explode()

(1)繼承org.apache.hadoop.hive.ql.UDF

(2)需要實(shí)現(xiàn)evaluate函數(shù);evaluate函數(shù)支持重載酪耳;

  • 注意事項(xiàng)

(1)UDF必須要有返回類型浓恳,可以返回null刹缝,但是返回類型不能為void;

(2)UDF中常用Text/LongWritable等類型颈将,不推薦使用java類型梢夯;

2、自定義函數(shù)開發(fā)
第一步:創(chuàng)建maven java 工程晴圾,并導(dǎo)入jar包

<pre class="md-fences md-end-block ty-contain-cm modeLoaded" lang="xml" style="box-sizing: border-box; overflow: visible; font-family: var(--monospace); font-size: 0.9em; display: block; break-inside: avoid; text-align: left; white-space: normal; background-image: inherit; background-position: inherit; background-size: inherit; background-repeat: inherit; background-attachment: inherit; background-origin: inherit; background-clip: inherit; background-color: rgb(248, 248, 248); position: relative !important; border: 1px solid rgb(231, 234, 237); border-radius: 3px; padding: 8px 4px 6px; margin-bottom: 15px; margin-top: 15px; width: inherit; color: rgb(51, 51, 51); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;" spellcheck="false" mdtype="fences" cid="n937"> <repositories>
<repository>
<id>cloudera</id>
<url>https://repository.cloudera.com/artifactory/cloudera-repos/</url>
</repository>
</repositories>

<dependencies>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-client</artifactId>
<version>2.6.0-mr1-cdh5.14.2</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>2.6.0-cdh5.14.2</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-hdfs</artifactId>
<version>2.6.0-cdh5.14.2</version>
</dependency>
?
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-mapreduce-client-core</artifactId>
<version>2.6.0-cdh5.14.2</version>
</dependency>
<dependency>
<groupId>org.apache.hive</groupId>
<artifactId>hive-exec</artifactId>
<version>1.1.0-cdh5.14.2</version>
</dependency>
<dependency>
<groupId>org.apache.hive</groupId>
<artifactId>hive-jdbc</artifactId>
<version>1.1.0-cdh5.14.2</version>
</dependency>
<dependency>
<groupId>org.apache.hive</groupId>
<artifactId>hive-cli</artifactId>
<version>1.1.0-cdh5.14.2</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.0</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
<encoding>UTF-8</encoding>

</configuration>
</plugin>
</plugins>
</build></pre>

第二步:開發(fā)java類繼承UDF颂砸,并重載evaluate 方法

<pre class="md-fences md-end-block ty-contain-cm modeLoaded" lang="java" style="box-sizing: border-box; overflow: visible; font-family: var(--monospace); font-size: 0.9em; display: block; break-inside: avoid; text-align: left; white-space: normal; background-image: inherit; background-position: inherit; background-size: inherit; background-repeat: inherit; background-attachment: inherit; background-origin: inherit; background-clip: inherit; background-color: rgb(248, 248, 248); position: relative !important; border: 1px solid rgb(231, 234, 237); border-radius: 3px; padding: 8px 4px 6px; margin-bottom: 15px; margin-top: 15px; width: inherit; color: rgb(51, 51, 51); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;" spellcheck="false" mdtype="fences" cid="n939">package com.kkb.udf.MyUDF;
?
public class MyUDF extends UDF {
public Text evaluate(final Text s) {
if (null == s) {
return null;
}
//返回大寫字母
return new Text(s.toString().toUpperCase());
}
}</pre>

第三步:項(xiàng)目打包
  • 使用maven的package進(jìn)行打包

  • 將我們打包好的jar包上傳到node03服務(wù)器的/kkb/install/hive-1.1.0-cdh5.14.2/lib 這個(gè)路徑下

第四步:添加我們的jar包
  • 重命名我們的jar包名稱

<pre class="md-fences md-end-block ty-contain-cm modeLoaded" lang="shell" style="box-sizing: border-box; overflow: visible; font-family: var(--monospace); font-size: 0.9em; display: block; break-inside: avoid; text-align: left; white-space: normal; background-image: inherit; background-position: inherit; background-size: inherit; background-repeat: inherit; background-attachment: inherit; background-origin: inherit; background-clip: inherit; background-color: rgb(248, 248, 248); position: relative !important; border: 1px solid rgb(231, 234, 237); border-radius: 3px; padding: 8px 4px 6px; margin-bottom: 15px; margin-top: 15px; width: inherit; color: rgb(51, 51, 51); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;" spellcheck="false" mdtype="fences" cid="n950">cd /kkb/install/hive-1.1.0-cdh5.14.2/lib
mv original-day_hive_udf-1.0-SNAPSHOT.jar udf.jar</pre>

  • hive的客戶端添加我們的jar包

<pre class="md-fences md-end-block ty-contain-cm modeLoaded" lang="sql" style="box-sizing: border-box; overflow: visible; font-family: var(--monospace); font-size: 0.9em; display: block; break-inside: avoid; text-align: left; white-space: normal; background-image: inherit; background-position: inherit; background-size: inherit; background-repeat: inherit; background-attachment: inherit; background-origin: inherit; background-clip: inherit; background-color: rgb(248, 248, 248); position: relative !important; border: 1px solid rgb(231, 234, 237); border-radius: 3px; padding: 8px 4px 6px; margin-bottom: 15px; margin-top: 15px; width: inherit; color: rgb(51, 51, 51); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;" spellcheck="false" mdtype="fences" cid="n954">0: jdbc:hive2://node03:10000> add jar /kkb/install/hive-1.1.0-cdh5.14.2/lib/udf.jar;</pre>

第五步:設(shè)置函數(shù)與我們的自定義函數(shù)關(guān)聯(lián)

<pre class="md-fences md-end-block ty-contain-cm modeLoaded" lang="sql" style="box-sizing: border-box; overflow: visible; font-family: var(--monospace); font-size: 0.9em; display: block; break-inside: avoid; text-align: left; white-space: normal; background-image: inherit; background-position: inherit; background-size: inherit; background-repeat: inherit; background-attachment: inherit; background-origin: inherit; background-clip: inherit; background-color: rgb(248, 248, 248); position: relative !important; border: 1px solid rgb(231, 234, 237); border-radius: 3px; padding: 8px 4px 6px; margin-bottom: 15px; margin-top: 15px; width: inherit; color: rgb(51, 51, 51); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;" spellcheck="false" mdtype="fences" cid="n956">0: jdbc:hive2://node03:10000> create temporary function touppercase as 'com.kkb.udf.MyUDF';</pre>

第六步:使用自定義函數(shù)

<pre class="md-fences md-end-block ty-contain-cm modeLoaded" lang="sql" style="box-sizing: border-box; overflow: visible; font-family: var(--monospace); font-size: 0.9em; display: block; break-inside: avoid; text-align: left; white-space: normal; background-image: inherit; background-position: inherit; background-size: inherit; background-repeat: inherit; background-attachment: inherit; background-origin: inherit; background-clip: inherit; background-color: rgb(248, 248, 248); position: relative !important; border: 1px solid rgb(231, 234, 237); border-radius: 3px; padding: 8px 4px 6px; margin-bottom: 15px; margin-top: 15px; width: inherit; color: rgb(51, 51, 51); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;" spellcheck="false" mdtype="fences" cid="n958">0: jdbc:hive2://node03:10000>select tolowercase('abc');</pre>

  • hive當(dāng)中如何創(chuàng)建永久函數(shù)

  • 在hive當(dāng)中添加臨時(shí)函數(shù),需要我們每次進(jìn)入hive客戶端的時(shí)候都需要添加以下死姚,退出hive客戶端臨時(shí)函數(shù)就會(huì)失效人乓,那么我們也可以創(chuàng)建永久函數(shù)來讓其不會(huì)失效

  • 創(chuàng)建永久函數(shù)

<pre class="md-fences md-end-block ty-contain-cm modeLoaded" lang="sql" style="box-sizing: border-box; overflow: visible; font-family: var(--monospace); font-size: 0.9em; display: block; break-inside: avoid; text-align: left; white-space: normal; background-image: inherit; background-position: inherit; background-size: inherit; background-repeat: inherit; background-attachment: inherit; background-origin: inherit; background-clip: inherit; background-color: rgb(248, 248, 248); position: relative !important; border: 1px solid rgb(231, 234, 237); border-radius: 3px; padding: 8px 4px 6px; margin-bottom: 15px; margin-top: 15px; width: inherit; color: rgb(51, 51, 51); font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;" spellcheck="false" mdtype="fences" cid="n966">-- 1、指定數(shù)據(jù)庫知允,將我們的函數(shù)創(chuàng)建到指定的數(shù)據(jù)庫下面
0: jdbc:hive2://node03:10000>use myhive;
?
-- 2撒蟀、使用add jar添加我們的jar包到hive當(dāng)中來
0: jdbc:hive2://node03:10000>add jar /kkb/install/hive-1.1.0-cdh5.14.2/lib/udf.jar;
?
-- 3、查看我們添加的所有的jar包
0: jdbc:hive2://node03:10000>list jars;
?
-- 4温鸽、創(chuàng)建永久函數(shù)保屯,與我們的函數(shù)進(jìn)行關(guān)聯(lián)
0: jdbc:hive2://node03:10000>create function myuppercase as 'com.kkb.udf.MyUDF';
?
-- 5、查看我們的永久函數(shù)
0: jdbc:hive2://node03:10000>show functions like 'my';
?
-- 6涤垫、使用永久函數(shù)
0: jdbc:hive2://node03:10000>select myhive.myuppercase('helloworld');
?
-- 7姑尺、刪除永久函數(shù)
0: jdbc:hive2://node03:10000>drop function myhive.myuppercase;
?
-- 8、查看函數(shù)
show functions like 'my
';</pre>

?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請聯(lián)系作者
  • 序言:七十年代末蝠猬,一起剝皮案震驚了整個(gè)濱河市切蟋,隨后出現(xiàn)的幾起案子,更是在濱河造成了極大的恐慌榆芦,老刑警劉巖柄粹,帶你破解...
    沈念sama閱讀 218,755評論 6 507
  • 序言:濱河連續(xù)發(fā)生了三起死亡事件,死亡現(xiàn)場離奇詭異匆绣,居然都是意外死亡驻右,警方通過查閱死者的電腦和手機(jī),發(fā)現(xiàn)死者居然都...
    沈念sama閱讀 93,305評論 3 395
  • 文/潘曉璐 我一進(jìn)店門崎淳,熙熙樓的掌柜王于貴愁眉苦臉地迎上來堪夭,“玉大人,你說我怎么就攤上這事拣凹∩” “怎么了?”我有些...
    開封第一講書人閱讀 165,138評論 0 355
  • 文/不壞的土叔 我叫張陵嚣镜,是天一觀的道長爬迟。 經(jīng)常有香客問我,道長祈惶,這世上最難降的妖魔是什么雕旨? 我笑而不...
    開封第一講書人閱讀 58,791評論 1 295
  • 正文 為了忘掉前任扮匠,我火速辦了婚禮,結(jié)果婚禮上凡涩,老公的妹妹穿的比我還像新娘棒搜。我一直安慰自己,他們只是感情好活箕,可當(dāng)我...
    茶點(diǎn)故事閱讀 67,794評論 6 392
  • 文/花漫 我一把揭開白布力麸。 她就那樣靜靜地躺著,像睡著了一般育韩。 火紅的嫁衣襯著肌膚如雪克蚂。 梳的紋絲不亂的頭發(fā)上,一...
    開封第一講書人閱讀 51,631評論 1 305
  • 那天筋讨,我揣著相機(jī)與錄音埃叭,去河邊找鬼。 笑死悉罕,一個(gè)胖子當(dāng)著我的面吹牛赤屋,可吹牛的內(nèi)容都是我干的。 我是一名探鬼主播壁袄,決...
    沈念sama閱讀 40,362評論 3 418
  • 文/蒼蘭香墨 我猛地睜開眼类早,長吁一口氣:“原來是場噩夢啊……” “哼!你這毒婦竟也來了嗜逻?” 一聲冷哼從身側(cè)響起涩僻,我...
    開封第一講書人閱讀 39,264評論 0 276
  • 序言:老撾萬榮一對情侶失蹤,失蹤者是張志新(化名)和其女友劉穎栈顷,沒想到半個(gè)月后逆日,有當(dāng)?shù)厝嗽跇淞掷锇l(fā)現(xiàn)了一具尸體,經(jīng)...
    沈念sama閱讀 45,724評論 1 315
  • 正文 獨(dú)居荒郊野嶺守林人離奇死亡萄凤,尸身上長有42處帶血的膿包…… 初始之章·張勛 以下內(nèi)容為張勛視角 年9月15日...
    茶點(diǎn)故事閱讀 37,900評論 3 336
  • 正文 我和宋清朗相戀三年屏富,在試婚紗的時(shí)候發(fā)現(xiàn)自己被綠了。 大學(xué)時(shí)的朋友給我發(fā)了我未婚夫和他白月光在一起吃飯的照片蛙卤。...
    茶點(diǎn)故事閱讀 40,040評論 1 350
  • 序言:一個(gè)原本活蹦亂跳的男人離奇死亡,死狀恐怖噩死,靈堂內(nèi)的尸體忽然破棺而出颤难,到底是詐尸還是另有隱情,我是刑警寧澤已维,帶...
    沈念sama閱讀 35,742評論 5 346
  • 正文 年R本政府宣布行嗤,位于F島的核電站,受9級特大地震影響垛耳,放射性物質(zhì)發(fā)生泄漏栅屏。R本人自食惡果不足惜飘千,卻給世界環(huán)境...
    茶點(diǎn)故事閱讀 41,364評論 3 330
  • 文/蒙蒙 一、第九天 我趴在偏房一處隱蔽的房頂上張望栈雳。 院中可真熱鬧护奈,春花似錦、人聲如沸哥纫。這莊子的主人今日做“春日...
    開封第一講書人閱讀 31,944評論 0 22
  • 文/蒼蘭香墨 我抬頭看了看天上的太陽蛀骇。三九已至厌秒,卻和暖如春,著一層夾襖步出監(jiān)牢的瞬間擅憔,已是汗流浹背鸵闪。 一陣腳步聲響...
    開封第一講書人閱讀 33,060評論 1 270
  • 我被黑心中介騙來泰國打工, 沒想到剛下飛機(jī)就差點(diǎn)兒被人妖公主榨干…… 1. 我叫王不留暑诸,地道東北人蚌讼。 一個(gè)月前我還...
    沈念sama閱讀 48,247評論 3 371
  • 正文 我出身青樓,卻偏偏與公主長得像屠列,于是被迫代替她去往敵國和親啦逆。 傳聞我的和親對象是個(gè)殘疾皇子,可洞房花燭夜當(dāng)晚...
    茶點(diǎn)故事閱讀 44,979評論 2 355

推薦閱讀更多精彩內(nèi)容