當(dāng)處理數(shù)據(jù)超過10w條趁猴,文件超過50M刊咳,你需要考慮一款新的數(shù)據(jù)處理軟件,MS旗下Access應(yīng)該是最好上手儡司,最便捷的一款娱挨。當(dāng)然,數(shù)據(jù)處理的工具不止這款捕犬,R跷坝、Python、Spoon同樣值得擁有碉碉!但是單單從DA的角度來(lái)看柴钻,我們關(guān)注的點(diǎn)在Analysis,難點(diǎn)BigData垢粮,用MS旗下Access足以將Data從50M10w條里瘦身顿颅。一起來(lái)學(xué)習(xí)一下!
PS:當(dāng)然足丢,SQL結(jié)構(gòu)化查詢規(guī)則較多粱腻,尤其MS旗下Access,先Mark斩跌,后續(xù)補(bǔ)上Python&R(又給自己挖坑 ><)
錄入數(shù)據(jù)
打開Access绍些,外部數(shù)據(jù)選項(xiàng)卡插入BigBoss(需要瘦身的data源),我保存的是excel文件耀鸦,也可以是text等柬批,你可以導(dǎo)入數(shù)據(jù)啸澡,也可以創(chuàng)建表鏈接。
數(shù)據(jù)瘦身
創(chuàng)建查詢氮帐,就可以開始你數(shù)據(jù)瘦身之路嗅虏。
點(diǎn)擊“user”表,你可以看到倒入的數(shù)據(jù)明細(xì)上沐;
屏幕右下角皮服,點(diǎn)擊SQL,調(diào)出查詢界面参咙,在這里龄广,你可以開始SQL查詢語(yǔ)句。
數(shù)據(jù)處理
作為一名DA蕴侧,我們是結(jié)果導(dǎo)向型的择同,我們看到data的第一個(gè)想法是:我們能夠從數(shù)據(jù)中挖出什么緯度。净宵。敲才。這個(gè)也是我一直在思考的問題。择葡。以下就我研究的一些緯度做簡(jiǎn)單分享归斤。
一、地理分布
SELECT provice_name , count(id) as times
FROM user
GROUP BY provice_name;
REULT:可以用ORDER BY 降序(默認(rèn)ASC)刁岸,也可以手動(dòng)降序
SELECT provice_name , count(id) as times
FROM user
GROUP BY provice_name
ORDER BY times DESC
;
另脏里,分組的條件查詢不可以用WHERE,要用HAVING虹曙。
二迫横、轉(zhuǎn)化率
成功為1,失敗為0
SELECT level_id,
count(level_success) as num,
sum(level_success) as suc,
suc/num as suc_rate
FROM user
group by level_id
;
三酝碳、留存
一天一張表(結(jié)構(gòu)完全相同)
去重矾踱,值為1,統(tǒng)計(jì)頻次
select distinct date4.ip,count(*) as how_many,1 AS has_left from date4 group by date4.ip
;
留存用戶統(tǒng)計(jì):
select n.d as 日期, n.newuser as 新增用戶, l.has_left as 次留用戶, has_left/newuser as 次日留存率
from(
select '0803' as d,count(1) as newuser from (SELECT t1.ip FROM date3 t1 group by t1.ip)
) n left join(
select '0803' as d,count(1) as has_left from (select t1.ip from date3 t1 inner join date4 t2 on t1.ip = t2.ip group by t1.ip)
) l on n.d = l.d
;